For one thing, it's not a real score; they judged the results themselves and Putnam judges are notoriously tough. There was not a single 8 on the problem they claim partial credit for (or any partial credit above a 2) amongst the top 500 humans. https://kskedlaya.org/putnam-archive/putnam2024stats.html.
For another thing, the 2024 Putnam problems are in their RL data.
Also, it's very unclear how these competitions consisting of problems designed to have clear-cut answers and be solved by (well-prepared) humans in an hour will translate to anything else.
> Curating Cold Start RL Data: We constructed our initial training data through the following
process:
> 1. We crawled problems from Art of Problem Solving (AoPS) contests
, prioritizing math
olympiads, team selection tests, and post-2010 problems explicitly requiring proofs, total-
ing 17,503 problems.
> Why do you think that the 2024 Putnam programs that they used to test were in the training data?
Putnam solutions can be found multiple places online: https://kskedlaya.org/putnam-archive/, https://artofproblemsolving.com/community/c3249_putnam. These could have appeared in the training of the base LLM DeepSeek-V3.2-Exp or as problems in the training set - they do not give further detail on what problems they selected from AOPS and as the second link gives they are there.
"OpenAI uses the following robots.txt tags to enable webmasters to manage how their sites and content work with AI."
Then for GPT it says:
"GPTBot is used to make our generative AI foundation models more useful and safe. It is used to crawl content that may be used in training our generative AI foundation models. Disallowing GPTBot indicates a site’s content should not be used in training generative AI foundation models."
My read is that they are describing functionality for site owners to provide input about what the site owner thinks should happen. OpenAI is not promising that is what WILL happen, even in the narrow context of that specific bot.
What makes you think the secrets are small enough to fit inside people's heads, and aren't like a huge codebase of data scraping and filtering pipelines, or a DB of manual labels?
> Poor Sam Altman, 300B worth of trade secrets bought out from under him for a paltry few hundred million
Sorry, you don't lose people when you treat them well. Add to that Altman's penchant for organisational dysfunction and the (in part resulting) illiquidity of OpenAI's employees' equity-not-equity and this makes a lot of sense. Broadly, it's good for the American AI ecosystem for this competition for talent to exist.
In retrospect, I wonder if the original ethos of the non-profit structure of OpenAI was a scam from the get go, or just woefully naive. And to emphasize, I'm not talking just about Altman.
That is, when you create this cutting edge, powerful tech, it turns out that people are willing to pay gobs of money for it. So if somehow OpenAI had managed to stay as a non-profit (let's pretend training didn't cost a bajillion dollars), they still would have lost all of their top engineers to deeper pockets if they didn't pursue an aggressive monetization strategy.
That's why I want to gag a little when I hear all this flowery language about how AI will cure all these diseases and be a huge boon to humanity. Let's get real - people are so hyped about this because they believe it will make them rich. And it most likely will, and to be clear, I don't blame them. The only thing I blame folks for is trying to wrap "I'd like to get rich" goals in moralistic BS.
It wasn't exactly a scam, it's just nobody thought it'd be worth real money that fast, so the transition from noble venture to cash grab happened faster than expected.
> wonder if the original ethos of the non-profit structure of OpenAI was a scam from the get go, or just woefully naive
Based on behaviour, it appears they didn't think they'd do anything impactful. When OpenAI accidentally created something important Altman immediately (a) actually got involved to (b) reverse course.
> if somehow OpenAI had managed to stay as a non-profit (let's pretend training didn't cost a bajillion dollars), they still would have lost all of their top engineers to deeper pockets if they didn't pursue an aggressive monetization strategy
I'm not so sure. OpenAI would have held a unique position as both first mover and moral arbiter. That's a powerful place to be, albeit not a position Silicon Valley is comfortable or competent in.
I'm also not sure pursuing monetisation requires a for-profit structure. That's more a function of the cost of training, though again, a licensing partnership with, I don't know, Microsoft, would alleviate that pressure without requiring giving up control.
I'm skeptical that OpenAI was ever feasible as a nonprofit under it's original mission, which was:
> Our goal is to advance digital intelligence in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial return. Since our research is free from financial obligations, we can better focus on a positive human impact.
As soon as the power of AI became apparent, everyone wanted (and in some ways, needed) to make bank. This would have been true even if the original training costs weren't so high.
