We can't just assume that hackers around the world have 80GB NVidia GPUs laying around to run high-end, usable LLMs. And we certainly can't expect people to pay Amazon Web Services for a rented GPU, those things cost significant amounts of money.
Sticking to basic automation tools we have for spam filtering and automated reading of messages and the like, lets start with Reddit-level tools of just banning, blocking, users. And a global effort that helps kill spam accounts and sockpuppets.
It's not a bad start, but it's missing "trolling/shitposting".
(I'm hardly advocating that people start relying on openai for moderation, rather it would be great if people started figuring out how to do this locally with the tools we have. Their bias against chess players is pretty despicable)
A lot of the tech/approaches are already used elsewhere. Some ideas
1. Require university or employer emails to register (Blind uses this). Still post via alias but have a verified root email tied to a person IRL, would really cut down on the trolling and spam and nonsense.. NEETs might be out of luck but, well, their domain is the imageboard and that's not exactly what you want on usenet most of the time..
2. Require some hashcash style calculation to prove the value of a post via computation. Use an algorithm which is ASIC resistant - maybe Ethereum's old PoW?
3. as you mentioned AI - Moderation endpoints, it would be interesting to see if that can be created from LLaMA or such (OpenAI has this available but no reason to become dependent on them). People have mostly been looking at them for generation but as a classifier they also may fit the bill
Assuming moderation can be automated using the recent advancements in AI