I'd then consider going freegan and eating whatever the grocery store is throwing out, that I don't subsequently have to cook.

This is a fun question but only because it's pretend. My family lived in our car one summer and that wasn't so fun. My dad, bless his heart, would buy huge packages of sandwich cookies and I hate them to this day. We called them "shipping cookies" because he bought them for every long trip since they were dirt cheap.

On the other hand, we roasted hot dogs on long sticks in firepits at the park and I was too young to realize we were desperate so that was great fun.

Now, if I could afford it I would load my backpack with all the things I have to limit because they are too dense--nuts, dried fruit, jerky and meat sticks/sausage, along with some carrots and fruits that can take squeezing such as a grapefruit. I assume I'd spend the day walking around so I could burn up those rich foods.

If I couldn't afford the above I'd have to walk around the supermarket and select foods in a compromise of money vs. paleo vs. weight/volume.

Canned fish - sardines. This is what I typically eat when I'm out & about & get hungry. A tin is like 99 cents and full of good things (skin and bones). I know there's an issue w/ BPA in most tins, but it's possible to get clean cans.

Pigeons and squirrels?

I think a combination dumpster diving, foraging, buying food from a store and trying to get food from the food bank would sustain you. I'm definitely not above dumpster diving. I think having the tools you need to catch/collect/cook food in your pack would be better than filling the pack with food. I would get a bag of jerky, canned sardines and a jar of coconut oil (or ghee if coconut oil was too expensive) to give me the energy I needed to get my hunt and gather on. Butter, even in winter, might not last in NC due to warmer weather. I think I would try to avoid eating squirrels, pigeons and raccoon because of their diseases and parasites. Here's a quick list of foods off the top of my head I could eat in my area, that I can already easily recognize, gather/catch and know how to prepare. I live in North Carolina.
Spring + Summer:

I've always said that nuts and nut butter would be my choice in this situation. You get the most bang for your buck when it comes to overall "healthy" calories per serving.

the dumpster beside the best steakhouse in town?

Squirrel sashimi.

Beans for certain.

I live in San Diego, so I would try to figure out which seaweeds are edible. (I think they all are, but I don't know.) I'd also look into eating mussels and other critters in the tide pools, and maybe try to catch some minnows with some kind of small net. I think there are some edible plants throughout the mountain and canyon areas, so I'd try to figure out what's what in that regard and hone my foraging skills. Maybe also try to catch some rabbits or squirrels if possible — though I'd want to figure out how to cook them. And worms. Maybe grasshoppers.

In terms of what I'd buy from the store, I think eggs are a good choice. No need to cook them. Just crack them open, let the white run through my fingers while snagging the yolk and eat it.

Beef jerky is unfortunately expensive; some kind of salami might be a better option for meat.

Fruits and vegetables — whatever's on sale. Nuts, too.

I would make fires or make a small camping stove (pretty easy--can be done with a soda can or cat food tin and alcohol fuel is cheap). Then: rice, beans, canned sardines, and whatever vegetables I could scrounge. Eventually, provided my health was good, I'd try and move to a more rural area so I could trap small game without hassle. I definitely wouldn't live in a city. I'd also offer work washing dishes at local churches/restaurants in exchange for a free meal (it worked this summer).

Kitty cats! Like in Mrs. Lovet's meat pies. Until she met Sweeny and improved the recipe. YUM!

When it comes to cheap, hard to beat rice and beans. Neither are 'kosher' paleo, but both are on the benign end of neolithic foods. Just need a fat source, coconut oil is too expensive. Butter might do it, again, sacrificing a bit of quality (foregoing pastured/organic).

If you have to eat out of a store, which the original question proposes, and you can't cook anything, then I'd go with a gallon of milk. And if I could afford it, a dozen eggs. Mix and drink. That's two days of food right there. Depends on what you call paleo about the milk, but I would not trust to eat raw meat out of the store (whereas I don't worry too much about eating my own home grown meat raw). I'm not really too concerned about salmonella from store eggs so would eat them raw.

Fresh vegetables from the store are too expensive unless you find a deal on something on the over age cart some stores have. Foraging would make better sense. If it's winter and you're in the north, then you stick with milk and eggs.

I would also consider high quality meat based dog food. I don't buy it now so don't know what the prices are, so it might be quite expensive, but it's also probably some of the biggest nutrition bang for your buck since they have to support a lactating bitch by USDA regulations.

I'd go with sardines, apples, canned salmon, canned chicken etc.,

That would suck BTW.

I'm usually a victim of mosquitos, but if I were homeless, I'd be sure it happened the other way around.

And I'd go on sample Sundays and try to sneakily get more than my fair share of sample cheese, meats and fruit. For free.

And dude, I wouldn't go to a grocery store. I'd go to a food pantry or soup kitchen.

One dozen eggs.

I always think it's interesting to see homeless people asking for food in a park. We have forgotten how to survive. If I ever was homeless I would hunt small game. Just cook it up and move on. Carry hunting and fire supplies.

I think obviously we are talking a cheap way to eat Paleo.

Dig for sales and stick with the basics. 50% nuts and fruit of various color, 50% lean meets. Add water and go for as long as you can. I've two months left on a trip before going home. With limited groceries and a little over $100 dollars a day to spend on food, I'll see how low I can go on three day increments once my 5 day supply of groceries are expended. This is nice.

I'd line up for the soup kitchen and ask them if they have any bone broth.

Nuts from the store. Its amazingly easy to open bag of nuts in the megashops and eat it there (ah... student times) :) Banana. Yogurt/Milk. Eggs are great, but since homeless, only raw ones are option and on the spot since they are not easy to carry around. Vitamin C powder.