In the first trimester avoid excess vitamin C (more than 1000mg/day) and magnesium (more than 600mg supplemental/day) because both have been linked to accidental miscarriage.
I learned after I had been taking it for years that 5-HTP shouldn't be taken during pregnancy because it can cause heart defects, so I stopped mid second trimester and luckily baby seemed okay on the heart scan done by ultrasound.
After that scare, I now wouldn't take any herb without the okay of my midwife, an herbalist, or a whole lot of googling, except for maybe red raspberry leaf infusion. Some are okay during some stages and problematic during others, so a quick phone call or email wouldn't hurt.
Certain green drinks (the one I normally take is Vitamineral Green) alongside all the great stuff contain things that might not be great for the baby in the first trimester like horsetail, the variety of seaweeds, and parsley juice. My nutritionist contacted the company on my behalf, and they said, it wouldn't have anything in great enough dosage to cause harm, but the recommended dosage can very wildly from 1 tsp. to 4 tblsp., so I chose err on side of caution with that one.
Personally though, the sudden onset of food aversions has directed me away from things that could be harmful, coffee went from being my favorite thing to horrid overnight, so if you stick to whole foods, I think you can trust yourself and your nose. It just gets a bit tricky when the flavors and smells are hidden in capsules and can't get fully vetted by your hyperactive sense of smell during pregnancy.