There are several foods considered to be generally "non allergenic" which means that MOST people don't experience any sensitivities. However, this doesn't mean it's absolutely perfect for everyone. Kiwi is often touted as non-allergenic, but it will land my sister a fast trip to the ER. Rice is considered to be a safe food on most elimination diets but I have a friend whose daughter is extremely allergic to rice. This is the rice allergy I know of personally and it's probably pretty rare, but these are examples of why you should take a very personal approach to food sensitivies.
Elimination diets are a great tool, especially if used in combination with a detailed journal, as well as determination and a lot of patience. The best chance of truly identifying foods that you're sensitive to is the slow route -- this means adding in just one or two new foods a week. That's a challenge for some people, because they get bored or frustrated with their limited choices and jump ship. So you could do an elimination diet in stages instead of making it a year-long experiment.
Create a "base diet" of just a few foods. You can either choose those recommended by your specialist or you can research to find a list of the least allergenic foods or you can use your own experiences with foods to come up with a plan (or some combination thereof). Basically, you want to pick foods that are the least likely to be problematic for you. The general idea is to eat only these foods for a while, to let your body "clear" and reset. Often food allergies are masked because we eat them often enough that we don't notice the signals our bodies send. Instead, having occasional headaches or intestinal upsets or otherwise feeling slightly unwell becomes our "normal" and we don't realize it's because we've exposed ourselves to an allergen. Once we've removed these offenders for a while then it's more likely we'll get a more noticeable reaction if we add them back into our diet later.
Don't add too many foods too fast or it will be difficult to pinpoint what has caused issues for you. There is also the "bundled" approach that has worked for some people -- add small groups of foods at a time say you try five new foods each week. If you have any type of unpleasant effects then you set those foods aside until you can do a strict elimination, adding only one at a time from that group until you identify which one(s) are the culprits.
As for what might be causing issues in your current diet, it's hard to tell. Unless you are eating very clean it could be from one or more additives, preservatives, or even pesticide residues you may be consuming. Some people are very sensitive and even minute quantities of these substances can cause issues. Peanuts are a frequent problem for a lot of people, even those who don't have a serious allergy. Also, you say that you rarely eat grains, however, if it turns out that you do indeed have trouble properly digesting grain proteins, you should know that the polypeptide chains formed from improperly digested grains can linger in the body a long time, months in some cases of people with celiac disease. So you may need to go on a strict grain-free diet for six months and see if that improves some of your issues.
