Reducing caffeine has had the single biggest impact on my health of all my dietary changes, easily. Ironically it hasn't had much impact on my sleep (which was caused by a different problem), but it has made me a lot calmer, more clear-thinking and astute, less anxious, generally happier, much more reliable energy especially in the morning etc.
Unfortunately despite cutting my caffeine back to nothing on a regular basis, I still end up taking it occasionally due to often sleeping badly (as per today, when I woke up early and couldn't get back to sleep) and needing to still be able to work. I still fine caffeine horrifically addictive, in that if I consume some one day, I'll get a pounding headache if I don't have some the next day (so I have to repeatedly taper off), and it's not as though it's just me liking the taste of coffee since atm I'm just having half a caffeine pill when necessary.
Also I can confirm that one's ability to handle coffee does vary wildly, even within the same individual. Before, I used to be able to drink huge amounts of caffeine constantly without ill effects, but increasingly it would just make me jittery and sick.