OpenAI and DeepMind previously hired human superforecasters to evaluate major model releases
"OpenAI hired superforecasters to evaluate the GPT-4 release, and then DeepMind hired superforecasters later to look into Gemini. Before there was AI superforecasting, there was still a need for decisions about what to do with these models, how to release them, what policies to have. And so the Frontier Labs did find it in their interest to bring on the superforecasters."
About this episode
Nathan Labenz and Prakash Shankar host AI in the AM with guest Dan Schwartz, CEO of FutureSearch, discussing the breakthrough moment in AI forecasting capabilities. Schwartz claims AI forecasters have reached superhuman performance in the last six months, matching or exceeding top human superforecasters on multiple benchmarks including ForecastBench and Metaculus tournaments. Scott Alexander recently validated these claims in a post titled 'The AI Superforecasters Are Here.' FutureSearch has been making hundreds of real-money trades on prediction markets like Polymarket and Kalshi using pure forecasting edge, not arbitrage or market-making. The episode also covers a major $160 million grant to Jeffrey Irving's Resolution AI safety nonprofit from Coefficient Giving, representing the largest known funding to an independent safety organization. Schwartz reveals that both OpenAI and DeepMind previously hired human superforecasters to evaluate GPT-4 and Gemini releases respectively. The hosts discuss controversial statements from Roon about tool AI being outcompeted by autonomous agents that will eventually execute human value systems better than humans themselves. Schwartz argues forecasting provides the only renewable source of training data as models surpass human experts, since ground truth emerges simply by waiting for future events to resolve. The episode explores implications for AI safety, financial markets, decision-making, and the coming transition to AI systems that may reason about the future in fundamentally alien ways. The conversation also touches on Fable's improved code management capabilities and the hosts demonstrate their new voice AI co-host Q live on the stream.
Key takeaways
- FutureSearch CEO Dan Schwartz claims AI forecasting has reached superhuman performance in the last six months, matching or beating top human superforecasters on multiple benchmarks including ForecastBench and Metaculus.
- Jeffrey Irving's Resolution AI safety nonprofit received $160 million from Coefficient Giving, the largest known grant to an independent safety organization, with $108 million unconditional funding.
- Roon posted that tool AI will be outcompeted by autonomous agents believing they are moral agents who will eventually execute human value systems better than humans themselves.
- FutureSearch has made hundreds of profitable trades on Polymarket and Kalshi using pure forecasting edge, potentially demonstrating AI outperforming aggregated human crowd wisdom in prediction markets.
- OpenAI hired superforecasters to evaluate GPT-4 release and DeepMind did the same for Gemini, revealing frontier labs sought external forecasting expertise for critical deployment decisions.
- Schwartz argues forecasting provides the only unlimited renewable source of training data with verifiable ground truth as AI models surpass human expert performance across domains.
- The hosts demonstrate their new voice AI co-host Q live on stream, which uses OpenAI's bidirectional API with speaker diarization and can perform web searches and answer questions in real-time.