This is a general ramble response to your examples and thoughts. I don't know if I actually answered your specific question or not. Downvote away!
1) Physical tolerance vs physical benefit
Virtually anything in too great a quantity will be "bad". Eat enough meat and you'll barf. So you have to look at your body's physical tolerances for a food and see what the benefits are for approaching them, keeping in mind always that individual variance can be huge.
When it comes to fructose, there's varying levels of "safe" tolerances thrown out there. Alan Aragorn comes up with 50g/day. Kurt Harris comes up with 5-15g/day. The benefit from eating fructose doesn't really appear to be there, though - so why eat any? Yes, you can survive eating it in small amounts without apparent cost, but if there's no positive benefit, why consume any?
This gets complicated depending on the food you're talking about (maybe there are some nutrients coming along for the ride) but then you have to ask if you couldn't get those nutrients elsewhere.
Fiber is somewhat similar. Too much is definitively bad for your digestive tract. Some coming along with vegetables is tolerable. Beneficial? Necessary? I agree that turning it into "all fiber is universally deadly" would be misleading and wrong, but unnecessary is probably a fair conclusion.
So from a physical tolerance vs physical benefit perspective, the fundamental questions are:
a) What can I tolerate safely?
b) What do I actually need to consume?
2) Physical tolerance versus physiological/psychological response
This is where people get all bent out of shape. Eating a potato will not ruin your metabolism physically - yeah there will be the dreaded blood sugar rise and insulin response and whatnot, but your body doesn't shrivel up and die. It will adapt and you'd be fine.
The issue here is the potential secondary effect of hunger or craving. If you eat a potato, you set a precedent, and you might want to eat more potatoes. How each individual handles this situation is totally different, of course, and it would be absurd to claim that eating a single potato will turn you into a ravenous carb addict again. But I think people have a legitimate fear/aversion to the risk of engaging the carb/hunger/craving cycle once having broken out of it, and so you get potato fears. Also there's always the paleo orthodoxy grumbling about nightshades and cooking, so there's a constant thundercloud over anything "neolithic" that I think biases people.
Most people who end up here are on board with the thinking that you want to get your body into a place where it naturally handles itself without your conscious intervention, and so fear things which threaten to disrupt the mechanisms by which that regulation would take place. The result is a lot of ZC/VLC folks who might not need to be ZC/VLC at a strictly physical level, but who believe they have to be in order to maintain their diets mentally, which basically makes it true that they need to avoid high carb foods.
3) Why are you eating in the first place?
This goes back to the questions I put at the end of #1 - why are you eating?
Some people see eating as an experience, others as a fuel top off. What one person sees as a fun treat another sees as analogous to recreational drug use. When non-paleos ask me how I feel about missing out on all that delicious sugar, I usually respond with a snappy, "About the same as I feel about not taking heroin", which is always fun. The same I suppose is true within the paleo community, where some people see food as something more than fuel, and thus are willing to tolerate things the pure-fuel folks wouldn't (or are unwilling to tolerate the pure-fuel lifestyle).
4) Orthorexia is bad
I actually hate the term "orthorexia" because the diagnosis for it is basically "anyone who is health conscious from any perspective at all" (seriously, see the wiki for it), but at the same time I think there's a real danger of falling into that sort of dogmatic mental trap. It's fine to abstain from foods, but to positively assert them to be dangerous to everyone is a much different thing. I personally can't eat nuts, and so yes, for me "an ounce of almonds is bad" would be a true statement. However, I wouldn't attempt to universally justify that using their omega-6 content - it's an individual tolerance, not a universal human tolerance.