Looking on the Nutrition Data site and if the glycemic load really does indicate how much insulin a food produces in the body then you are right about the milk, but the same amount of cheese is less so:
100 grams of whole milk (3.25% milk fat) has an estimated glycemic load of 4.
100 grams of cream (half and half) has an estimated glycemic load of 3.
100 grams of cheddar cheese (hard) has an estimated glycemic load of 1.
100 grams of goats cheese (soft) has an estimated glycemic load of 1.
100 grams of butter (with or without salt) has an estimated glycemic load of 0.
ADDITION: 100 grams of heavy whipping cream has an estimated glycemic load of 0.
100 grams of soured, cultured cream (a.k.a. creme fraiche) cream has an estimated glycemic load of 2.
edit: but judging from the other answers and comments here, the GL of food is not the best thing to refer to in the case of insulinogenic factors.