Cavemen didn't. I suspect not wiping creates a self cleaning ass?
Not that I should be legitimizing this with an answer but I can't help myself.
When one poops in a squatting position there is less mess and less need to wipe. That said, people in third world countries that cannot afford toilet paper often use one of their hands to wipe if there is excess and in countries like India, because of that, there is a taboo against eating with your left hand, which is presumed to be the wiping hand.
first I thought of...
Which obviously lead me too
...someone is out there using this thing...right...now...live in fear...
Wiping with paper is totally modern. Obviously.
Most small settlements and camps would obligatorily lie near running water, streams, rivers, etc. A perfect place to clean oneself. Just the smell of what emerges from the anus, and the way it would attract all manner of flying and crawling creatures would, in time, be enough to show even the most primitive individual to have it be far from where he ate, slept and sexed. As well as the desire to remove the remnants of the aforementioned defecation from one's one external skin areas.
Leaves would be an obvious choice if water were not readily available.
Perhaps the need to wipe grows when we don underwear and pants (which maybe cavemen didn't do). And wiping with leaves could have provided a "teachable moment" for young cavechildren. Poison ivy? Stinging nettle? Bet they only tried those once....