I suspect this is going to get everyone thinking I've lost the plot, totally & utterly but it really does work and it is completely gluten free to boot.
There are a number of recipes out there that use a small quantity of mashed potato and plenty of flour which just didn't feel quite right for "an ingredient challenge", so I went with one that was almost all potato and a little almond flour/meal/ground almonds.
Retro Recipe - Lemon Potato Cake
I'll apologise for overcooking it hence it collapsing a touch, it is still a surprisingly light and fluffy cake. It's not particulary sweet. In fact it is only slightly sweet and slightly lemony. I've added a sour* lemon drizzle to ours (* made using Meyer lemons & no sugar) to enhance the lemon flavour but you could easily add any fruit or even melted chocolate whilst making the cake if you so desired.