Amazon and Vitacost both have a lot of paleo staples-- i.e. coconut oil-- and you can (and probably should!) order your grassfed and/or pastured meats and poultry directly from a farm, it will be less expensive and gets more money directly into the hands of a farmer instead of a corporation, too. You can check eatwild.com for farms near you but don't rule out other states, as even with shipping, it might be more economical to order from a farm that's further away.
For fruits/veggies-- it's best to stay local, but not necessarily from a grocery store-- check www.localharvest.org for CSAs, farmers markets and buying clubs near you. If nothing is local to you, consider making a trip once every few months to stock up on frozen organic produce from the nearest large town with larger grocery stores and/or warehouse clubs (i.e. Costco).
I live in the suburbs of a fairly large metropolitan area and I still do all of the above-- it's far more convenient (I almost never have to go to the grocery store!) and far more economical, too.
Hope that helps!