The answer would depend on your definition of paleo. Was our beloved grok making tasty cakes and muffins? No. So if your idea of paleo falls along those lines you're pretty much out of luck.
As cliff said they are relatively harmless. Many of us have left reenactment for "above all else do no harm" as the guiding light for our food choices. We eat things that are neolithic but are not harmful as best as we can tell. I would use aluminum free baking powder. Otherwise I say bake away!
No, they're not paleo. But if you want to consume them, go for it! I'll add a bit of baking powder to my paleo pancakes (which consist of coconut flour, flax seed, eggs, coc oil, pecan meal) since the stuff is pretty benign, especially in such small quantities.
The question SHOULD NOT be: "Is this paleo?"
The question SHOULD BE: "Is this going to have a detrimental effect on my health?"
If the answer is no, it's fair game! There is so much gray area here, so don't sweat the small stuff! Undue stress raises cortisol, which is something we're all trying to avoid! :-)
I agree with Shari in that neither are paleo in the strictest sense of the word.
Before I went paleo, I used to bake for family and friends quite often. When I go to visit or have friends over for coffee, I like to bake some "paleo" muffins. If nothing else, it is a good way to steer the conversation over to nutrition issues!
Having said that, the baking powder I use does contain cornstarch, which is definitely a no-no. I usually use only a small amount, so I don't get too worked up about it.