It was suggested that I could take them to small claims, but I've never done anything like that before, and looking into it seems that Amazon makes it rather hard and admin heavy to do.
I've basically written off KDP for all future publications, and most of my current stuff is for sale through all the other book retailers.
It's a real shame as Amazon is the heart of a lot of indie publishing and provides a good service but ultimately they don’t care about the individual.
I have seen stories of slightly bigger indie authors also raising hell on Twitter and getting various other people involved (BSFA in this case) to speak to Amazon. In that case I believe they got paid.
I’m still shocked that Amazon let me pay for ads for a novel that I would never get the sales money for. That hurt more than just being delisted.
If you took them to small claims court, my concern is that they might do something like ban you from future publishing on Amazon/Kindle -- either because you've become a nuisance, or as a warning to other authors.
(Which could be bad for you, considering they're a gatekeeper to your access to much of the market.)
My thinking is that a regulator could offer some protection against retaliation, because that's a basic part of regulation, when you have people reporting.
Or a lawyer might be able to figure out how to protect you. Which might involve doing particular things, a particular way, at particular times.
Maybe a regulator gives you more anti-retaliation protection.