OpenAI Researcher Claims Model Benchmarks Published Without Compute Controls Are Misleading
"I think a lot of the skepticism came from the benchmark grid that was published. Basically, whenever a new model is released, there is this benchmark grid where they show all these different benchmarks on the x-axis and then the performance of different models on the y-axis. And you can just compare different models. It's like a single number for a model on a single benchmark. And if you look on paper at the difference between 5.5 and 5.4 or or other models, it wasn't, it was an improvement, but it wasn't a huge improvement."
About this episode
On this episode of No Priors, host Sarah Guo interviews Noam Brown, an AI researcher at OpenAI who pioneered inference-time scaling and reasoning approaches. Brown argues forcefully that the AI industry has a broken model evaluation system that fails to account for test-time compute, making benchmark comparisons misleading and safety evaluations inadequate. He reveals that modern models like GPT-4.5 can think productively for weeks before performance plateaus on complex tasks, unlike earlier models that peaked quickly, creating a fundamental problem: the capability of a model is now a function of how much money you spend on inference, from $10 to $10 million. Brown discloses that OpenAI used an unreleased internal model to disprove the Erdős unit distance conjecture, a longstanding mathematics problem, at minimal cost, and that GPT-4.5 could likely do the same with proper scaffolding for $1,000 to $100,000. He warns that current AI safety frameworks from all major labs don't account for this scaling dynamic, meaning models could have dangerous latent capabilities that aren't being tested at sufficient compute budgets. Brown also reveals OpenAI deliberately discourages internal researchers from using advanced models to solve open scientific problems, preferring to focus on developing more capable models for public release. He remains skeptical of overnight intelligence explosion scenarios, arguing time itself is the fundamental bottleneck because models require extended compute periods to reach peak capability. The conversation covers Brown's personal use of models for tasks from poker solver development to tax advice, his belief that models still lack research taste, and his call for the industry to abandon single-number benchmark grids in favor of performance plotted against inference budget.
Key takeaways
- Brown revealed current AI safety evaluation frameworks don't account for test-time compute scaling, creating potential blind spots for dangerous capabilities at high inference budgets.
- Modern AI models continue improving for weeks of compute time before plateauing, far longer than earlier models, making full capability assessment impossible within typical release cycles.
- OpenAI used an unreleased internal model to disprove the Erdős unit distance conjecture at minimal cost, demonstrating significant latent mathematical capability.
- Brown argues standard AI benchmarks showing single-number comparisons are misleading because they don't control for inference compute, making GPT-4.5 appear only marginally better than GPT-4.4.
- OpenAI deliberately discourages internal researchers from using advanced models to solve open scientific problems, focusing instead on developing more capable models for public release.
- Brown remains skeptical of overnight intelligence explosion scenarios, arguing time is a fundamental bottleneck because models require extended periods to reach peak capability.
- He disclosed that GPT-4.5 and similar models likely have significant unexplored capability because researchers haven't tested them at very large inference budgets like $100,000 per task.