I went VLC back in August, like 20-40g per day, and then added back starches (rotating potatoes and white rice) a few months ago. I have continued to get more cut in that time frame. That sounds like a very similar situation as you.
EDIT: for Ambimorph per below comment request
I don't have either every day. But I will have seasoned panfried potatoes with my eggs for a weekend breakfast. Or a small bowl of rice with about 3 tablespoons of pasture butter with a couple hardbioled eggs and sardines for lunch at work. We also frequently add potato or rice as a side to our dinners in addition to whatever meat and veggies we are having. I always peel the potato with a skinner. By themselves, neither hold much flavor, but they are both great vehicles for good fats. I don't usually eat starch for PWO, but not because I think it's a bad idea. I just don't really ever desire to eat that right afterwards, so personally I haven't found it very practical.