OpenAI Deliberately Limits Internal Use of Models to Solve Open Research Problems
"We are trying to encourage people to not spend all their time just like going through all the mathematical open problems, physics problems, and just seeing, pushing the models to their limits to see what they can prove or disprove. Because we really think the focus should be on how do we make even more capable models? How can we get them out safely to the world as quickly as possible so that all the scientists in the world can use these models to solve the problems themselves?"
About this episode
On this episode of No Priors, host Sarah Guo interviews Noam Brown, an AI researcher at OpenAI who pioneered inference-time scaling and reasoning approaches. Brown argues forcefully that the AI industry has a broken model evaluation system that fails to account for test-time compute, making benchmark comparisons misleading and safety evaluations inadequate. He reveals that modern models like GPT-4.5 can think productively for weeks before performance plateaus on complex tasks, unlike earlier models that peaked quickly, creating a fundamental problem: the capability of a model is now a function of how much money you spend on inference, from $10 to $10 million. Brown discloses that OpenAI used an unreleased internal model to disprove the Erdős unit distance conjecture, a longstanding mathematics problem, at minimal cost, and that GPT-4.5 could likely do the same with proper scaffolding for $1,000 to $100,000. He warns that current AI safety frameworks from all major labs don't account for this scaling dynamic, meaning models could have dangerous latent capabilities that aren't being tested at sufficient compute budgets. Brown also reveals OpenAI deliberately discourages internal researchers from using advanced models to solve open scientific problems, preferring to focus on developing more capable models for public release. He remains skeptical of overnight intelligence explosion scenarios, arguing time itself is the fundamental bottleneck because models require extended compute periods to reach peak capability. The conversation covers Brown's personal use of models for tasks from poker solver development to tax advice, his belief that models still lack research taste, and his call for the industry to abandon single-number benchmark grids in favor of performance plotted against inference budget.
Key takeaways
- Brown revealed current AI safety evaluation frameworks don't account for test-time compute scaling, creating potential blind spots for dangerous capabilities at high inference budgets.
- Modern AI models continue improving for weeks of compute time before plateauing, far longer than earlier models, making full capability assessment impossible within typical release cycles.
- OpenAI used an unreleased internal model to disprove the Erdős unit distance conjecture at minimal cost, demonstrating significant latent mathematical capability.
- Brown argues standard AI benchmarks showing single-number comparisons are misleading because they don't control for inference compute, making GPT-4.5 appear only marginally better than GPT-4.4.
- OpenAI deliberately discourages internal researchers from using advanced models to solve open scientific problems, focusing instead on developing more capable models for public release.
- Brown remains skeptical of overnight intelligence explosion scenarios, arguing time is a fundamental bottleneck because models require extended periods to reach peak capability.
- He disclosed that GPT-4.5 and similar models likely have significant unexplored capability because researchers haven't tested them at very large inference budgets like $100,000 per task.