If you eat a highly nutritious diet with relatively few anti-nutrients, and keep your teeth clean in between 'doses' of sugars (I have also read a lot of data that suggests that frequency of sugar intake is a big factor in tooth decay - and that sugary liquids are much, much worse than solids), for plenty of people consumption of sugars doesn't matter much.
YMMV according to the natural hardness and quality of your teeth; I have hard, healthy teeth, it's probably both genetic (my parents have great teeth too, although my middle sister has awful ones) and because my mom ate well before and when pregnant with me and fed me pretty well as kids. I almost never go to the dentist, but did a few months ago for the first time in 8 years, just to see what's up and get a little tartar I had noticed removed. My teeth are doing great.
Starches make my mouth feel more 'dirty' than straight sugar, and give me an acidic taste in my mouth...