Yes, there is general rule: you need all of them.

Deficiency of single vitamin may lead to many different problems. See The Ultimate Supplemental PaleoHacks Thread for some suggestions.

About guessing the dose, there is a lot of experimentation involved. You need to see which group of vitamins work with specific condition you want to address and see which are typical therapeutic doses.

Vitamins are generally non toxic so the worst thing that can happen to you is that nothing happens. There are some combining rules, like that you should take D3 and K2 together, but that topic is too big and context dependent.

• Be wary of mega-dosing vitamins/minerals. Our bodies are steady-state equilibriums. If you push too much of X in, you eliminate or prevent Y from being absorbed (Iron and Zinc can interfere with each other, for example.)
• It's easier to over-do fat-soluble vitamins (e.g. Vitamins A & E), hard to over-do water-soluble ones (e.g. Vitamin C).
• A number of vitamins/minerals have narrow therapeutic ranges. Too little is deficiency, but too much is toxicity. (Selenium, for example.)

I have been pulling my hair out over supplements, too. I never took a vitamin in my life until this summer. I have bottles of D3, E, Zinc, Magnesium, Krill Oil, Fish Oil, Flax Oil, B12, B-complex, and CoQ10. Every day, I take 1 or 2 of something. Most days I take 1 or 2 D3 5000IU, but that's the only one I take so frequently. The others are maybe twice a week.

I am looking for K2 and PQQ but they don't carry it anywhere in my town. May have to order off the nets.

I eat lots of eggs, liver, bone broth, and veggies. I figure adding in the biggies a few times a week will give my body a shot at some things I might miss in my diet without overdoing it.

D3 and magnesium should be the supplement baseline for most people, and even then you don't need the former if you are mostly naked daily at a low latitude, or the latter if you eat a lot of almonds every day.

Supplements can be a good way to rapidly address a deficiency, but they're not completely necessary if a significant portion of your diet is comprised of organs, sea creatures, egg yolks and meat. There's also the issue of supplements being unbalanced to an extent that evolutionarily speaking, our bodies have never coped with it. If you want to, for example, address a zinc deficiency, zinc picolinate is better than nothing, but oysters are probably better than zinc picolinate because though they have a ton of zinc, they still come with iron and copper.