Some of the restrictions are mostly used by people who have particular problems such as autoimmmune disorders with the nightshades.
Paleo isn't about reenactment, it's about finding what's optimal. Evolution of humans with only the last 10000 years spent with agriculture points the way towards what is likely optimal. As far as peppers go, only the few people who traveled out of Asia and down the Americas coastline ever would have encountered peppers - they're new world crops, only introduced to most of our bloodlines (unless you're part South American native) a few hundred years ago.
Dairy would have been introduced to some of us and genes that allowed digestion of lactose @ 5000 years ago. This mainly applies to those of us with European (especially northern European or Mideastern ancestors). Most Africans (except Masai) and Asians are lactose intolerant.
But again, all this points the way. Dairy is suspect because it is of recent introduction. Some of us handle it, some don't, and it may be somewhat subtle when it's a problem. Because most of it have never been dairy free for any length of time, we don't recognize the discomfort as abnormal until we do a trial and it goes away. This is a good area for self experimentation.
What's good to eat (this is a short and abbreviated list simply based on what I personally am eating while doing a Whole30, your list may be different):
Salmon, shrimp, crab, cod, anchovies, sardines, tuna, beef, pork, chicken, lamb....
Eggs...
lettuce, basil, broccoli, cauliflower, beet greens (mostly people no problem with oxalates), cucumbers, dandelion greens, bok choy, cilantro, tomatoes (see no autoimmune problems), eggplant, peppers, green beans, squash, beets, carrots, sweet potatoes, taro, plantain....
Berries, cherries, peaches, plums, melons, all fruits if you don't have metabolic syndrome.
Avocado, coconut, lard, palm oil, olive oil, olives.....
Sorry, fingers cramping, list too long, didn't write.