I had to quit chocolate bars. I would suffer a really bad mood after eating them. Certain brands were worse than others, but all would affect me to some extent. Life without chocolate is not too bad for me but definitely better for those around me.
One of the benefits to eating "paleo" is that you can easily add or remove foods and quickly ascertain whether or not they adversely affect you. When I was on the SAD I would have never been able to correlate my moods with certain foods.