A company I worked for switched over to pre-recorded demos and everyone talked about how clever it was for the first few larger audiences. Then they made a mistake and replayed a clip during the same session and the audience chat blew up. You could see a dip in new users for days after the demo.
While I get the academic perspective of sharing these insights, this article comes across as corporate justifying/complaining that their model's score is lower than it should be on the leaderboards... by saying the leaderboards are wrong.
Or an even darker take is that its coorporate saying they won't prioritize eliminating hallucinations until the leaderboards reward it.
Yes, it's self-interested because they want to improve the leaderboards, which will help GPT-5 scores, but in the other hand, the changes they suggest seem very reasonable and will hopefully help everyone in the industry do better.
And I'm sure other people will complain if notice that changing the benchmarks makes things worse.
This very much confused me. Isn't the idea behind this movement that Europe doesn't want to be dependent on external companies for critical infrastructure? Won't this just be the equivalent of a shell company completely dependant on Amazon in the US for any future fixes or R&D?
My team is debating this exact question for a new product we have in early access. Ultimately we realized the issue early on, so even our plans option would include at-cost usage limits.
Quote: "I don’t see the value in discussing a “national” crisis in homebuilding oligopoly from which the 49 biggest metros are exempt."
I enjoyed most of this article, but it did slide into opinions periodically. This quote in particular stood out given the issues rural communities face regarding availability of competition in a lot of other services.
