The fat in dairy contains CLA - it's the most beneficial component in milk!! It's all stripped out in fat free dairy!
As long as you're getting your milk from grass fed cattle (which you should be, and the more raw the better) then the fat s where the good stuff is! Stearic acid produced by grass fed cattle is awesome for you and it is a saturated fat. Why would you want to drink low fat dairy anyway? It tastes like milk flavored water (gross).
Well, one thing we can usually agree on here is to avoid as much processing as possible. And creating low-fat milk requires additional processing and the end product is less nutritious (as the important micronutrients are in the fat as others have alluded to). If you were a small farmer producing your own milk, you'd drink full fat milk because that is what the cow gave you.
High fat dairy pros include (assuming grass-fed and/or raw/minimally processed): CLA, Vitamins A & D, Calcium, minimal lactose (sugar involved in intolerance), minimal casein (protein often involved in milk sensitivities), saturated fat, X factor (for all you WAP folks, Vit K2 for everybody else) and unoxidized cholesterol.
Low fat dairy cons include: high lactose, higher casein, oxidized cholesterol (milk powder added back to increase "body"), "fortified" with vitamins rather than naturally occurring, higher likelihood of additives (guar gum, carageenan, etc.)
The point isn't to get dairy, but to get fat. (Low-fat dairy is just liquid sugar plus whatever additives.) It can be difficult to get enough fat into your diet when you try to keep down your PUFA intake, with the way livestock have been bred to be extra lean, and the high expense of other healthy fats like coconut oil. So high-fat dairy like butter and cream can be a good source of saturated fat, as long as you aren't dairy intolerant.
Like others have said, when you take the fat out all you're left with is sugar water. I'm pretty intolerant of dairy, but I can have 16 oz heavy cream lattes with no issue, and my main cooking fat is butter. So that leads me to believe that most of the bad stuff is not in the dairy fat.