Extra Terrible Lines are indeed fun. We've had 9 months of development since then, though; maybe it would make sense to repeat those experiments twice a year.
IIRC Scott Alexander is doing something similar with his "AI draws nontrivial prompts" bet, and the difference to last year's results was striking.
Also, this really needs blinding, otherwise the temptation to show off one's sophistication and subtlety is big. Remember how oenologists consistently fail to distinguish between a USD 20 and a USD 2000 wine bottle when blinded.
IIRC Scott Alexander is doing something similar with his "AI draws nontrivial prompts" bet, and the difference to last year's results was striking.
Also, this really needs blinding, otherwise the temptation to show off one's sophistication and subtlety is big. Remember how oenologists consistently fail to distinguish between a USD 20 and a USD 2000 wine bottle when blinded.