I think this is the only known benchmark that actually compares different quants: https://oobabooga.github.io/benchmark.html

It's not really all that consistent, but larger models can be compressed more without as much loss.