What they mean is that in a couple of weeks there are going to be stories titled "LLMS NOW BETTER THAN HUMANS AT 2025 INTERNATIONAL MATH OLYMPIAD" (stories published as thinly-veiled investment solicitations) but in reality they're still shitty-- they've just had the answers fed in to be spit back out.

Companies would game metrics whenever they have the opportunity. What else is new?

I suppose what's new is that the models aren't as smart as their companies claimed.

Not really.