Have you seen older women in no-clothes tribes? well, I don't want to have my breast hanging till navel any time soon. It is painful to do any physical exercise without a proper protection. Good bra also helps the back and shoulders. I do take it off the moment I get back home, though.
In terms of underwear for women - we have pubic hair for a reason. It's for protection, same as the fat over pubic bone. No underwear (with pubic hair intact) make sense only with loose skirt kind of clothing, not tight, possibly irritating pinching pants. I don't even want to think about the health risks and discomfort of wearing no underwear (and I mean real, not a tiny little string), shaved pubes and tight pants.
in terms of male, there seems to be the same principle - protect from possible injuries, but without too much coverage that could rise the temperature.