Here are my thoughts, though there are other similar threads as well, i.e. on minimalist cooking.
One, get rid of teflon. It's got a lot of bad stuff in it, and in my mind isn't in line with the paleo lifestyle. Just my opinion. Instead, invest in good cast iron. My skillet set doubles as a dutch oven (one skillet makes the lid). Thus I don't need a pot or pan.
Two, if you want to exercise in a more paleo way, bodyweight is really all you need in my mind. If you get to a point where something like a pull-up bar seems good, then go for it. For some good simple exercises, checkout Hillfit.com by Chris Highcock. Great stuff for strength and endurance. Also check out http://www.wildmovement.com/ There are some good free resources there of ways to paleo exercise without investing in equipment.
Three, if you want to start barefooting, that's a really cheap and easy way to reconnect with the earth. My only pair of shoes are Vibram FiveFingers, so I get the barefoot feeling wherever I go. If you're into that sort of thing, they are a good investment.
Lastly, anything you get, make sure it is something you enjoy. For example, I love my Vibrams, and my skillets or ulu cutting board. They're simple and efficient, but most of all they enrich the process of living paleo. If some item won't do that, don't get it. Paleo should be about feeling good and living well.