I've been going through many of the same thoughts, and you really break down the struggle that one faces trying to live "off-the-grid" so to speak. You need land, and good land isn't cheap. Then you have to worry about sewage and water.
I really like the idea of living in a greenhouse, with possibly a makeshift outhouse and composting toilet. You can live there temporarily while you build a cabin or house. But once again, you need the land to build on.
It seems like the best temporary solution under $5000 without land, is buying a nice RV camper like a Ford Econoline series. But I think if you really had to, you could buy land under $1000 and build a small "Shelter" for a couple grand. There are nice tents with wood burning stoves you might consider.
I haven't thought of tents with wood burning stoves. That's a good idea as well, thank you. I was going to possibly look into composite toilets, i hadn't heard much about them before.