When I was eating SAD and running competitively I never noticed a reduction in sex drive. If anything I have always found that moderate exercise enhances it. Maybe you are going beyond the level that is optimum for health?
I did experience a dramatically-lowered sex drive once, when trying out a low-fat CRON-ish diet (Melissa mentioned that already). It's very obvious when it happens, not easy to miss. May take some time to recover if you are coming from a vegetarian diet.
Would a PaNu diet affect reproduction? Kurt Harris says:
Insulin is a phylogenetically old hormone. It is a biological messenger that in excess, is metabolically saying the following to your tissue and organs: "Go ahead and store energy, mature, reproduce and die." source article
Does such a message at the cellular level directly correspond to reproduction at the organism level? Doubtful, although you can find interesting research in that vein if you dig around:
Since the phenotype of leptin-deficient ob/ob and leptin-resistant db/db mice is also characterized by hypothalamic hypogonadism, deficiency of either leptin or insulin action in the CNS is sufficient to impair reproduction. source article
Would be interesting to know how much insulin affects sex drive.
EDIT: Here is an excerpt of some notes from GCBC I was just reading:
The Carbohydrate Hypothesis, III: Hunger and Satiety
- nutrition and fertility are closely linked
- Le Magnen's hypothesis implies that fertility is mainly influenced by the immediate availability of fuel, not by body fat stores
- experiments confirm: animals become infertile without food (regardless of body
fat); the more food, the more fertile
- insulin stores fuel (makes it unavailable): infusing insulin into animals shuts down their reproductive systems
- thus, low-carb has clear implications for infertility treatment
Fertility is not the same as sex drive obviously, but quite interesting nonetheless