I'm going to have to look into this more, try to find what studies they're basing their conclusions on. It was my understanding that the cancer-causing beta-carotene was the one found in vitamins, NOT whole foods. This paragraph from the article seems to support my recollection: "The findings also might explain why, in a decades-old clinical trial, more people who were heavily *supplemented* with beta-carotene ended up with lung cancer than did research participants who took no beta-carotene at all. The trial was ended early because of that unexpected outcome."
In fact, the researchers aren't really talking about the effects of beta-carotene in whole foods. And it appears the whole purpose of their study is to understand why the artificial beta-carotene in the test on male smokers and asbestos workers created a higher incidence of lung cancer. So they can genetically engineer crops with increased beta-carotene for populations who are lacking vitamin A in their diets.
What I take away from this article is that the beta-carotene in our carrots and sweet potatoes is in no way the same as the beta-carotene used in the study that caused an increase in lung cancer. There is a danger when we isolate vitamins and minerals from the whole foods they came from because, as the researchers themselves stated, we don't fully understand how all the properties in a whole food work together to make it nourishing.