Absolutely. Our body chemistry and the environment of invisible dirt and bacteria around us is hugely complex and I'm sure we don't understand the majority of it. In this context, aggressively trying to remove dirt from our diet is probably a) damaging, you'd need to use chemicals or artificial growing environments, b) unnecessary, as stated, our bodies clearly evolved able to deal with a bit of dirt in its diet.
My guess is that as we evolved with eating dirt as part of our lives we quite possibly have a 'soft dependency' on something in dirt and by agressively removing it from our diet we'll lose out on some particular trace part of our diet.
I don't think you need to go out of your way to add dirt to your diet though.. just don't scrub those vegetables so hard and get it 'naturally'.
(I'd also be concerned that if someone did specifically add it to their diet they'd get far too much of a particular kind of dirt that may be contaminated. I definitely wouldn't want to eat a teaspoon of dirt from my London city house back yard, but I'm more than happy eating a bit of dirt attached to a potato that was grown organically in a field in the countryside)
