Forecasting identified as renewable source of training data as models surpass human experts
"You basically have a completely limitless set of extremely hard, basically impossible questions where you get exact ground truth. And there is no other eval like this. If you want to improve a coding harness, you just need to have more and more hard coding problems that are not in the training data for which you can say this is definitely the correct answer. Human experts who are trying to make evals are finding that they are not smarter than the things being trained anymore. You need something where there's a correct answer and the model can't figure it out. Forecasting, I think, is the only completely and utterly renewable source of this."
About this episode
Nathan Labenz and Prakash Shankar host AI in the AM with guest Dan Schwartz, CEO of FutureSearch, discussing the breakthrough moment in AI forecasting capabilities. Schwartz claims AI forecasters have reached superhuman performance in the last six months, matching or exceeding top human superforecasters on multiple benchmarks including ForecastBench and Metaculus tournaments. Scott Alexander recently validated these claims in a post titled 'The AI Superforecasters Are Here.' FutureSearch has been making hundreds of real-money trades on prediction markets like Polymarket and Kalshi using pure forecasting edge, not arbitrage or market-making. The episode also covers a major $160 million grant to Jeffrey Irving's Resolution AI safety nonprofit from Coefficient Giving, representing the largest known funding to an independent safety organization. Schwartz reveals that both OpenAI and DeepMind previously hired human superforecasters to evaluate GPT-4 and Gemini releases respectively. The hosts discuss controversial statements from Roon about tool AI being outcompeted by autonomous agents that will eventually execute human value systems better than humans themselves. Schwartz argues forecasting provides the only renewable source of training data as models surpass human experts, since ground truth emerges simply by waiting for future events to resolve. The episode explores implications for AI safety, financial markets, decision-making, and the coming transition to AI systems that may reason about the future in fundamentally alien ways. The conversation also touches on Fable's improved code management capabilities and the hosts demonstrate their new voice AI co-host Q live on the stream.
Key takeaways
- FutureSearch CEO Dan Schwartz claims AI forecasting has reached superhuman performance in the last six months, matching or beating top human superforecasters on multiple benchmarks including ForecastBench and Metaculus.
- Jeffrey Irving's Resolution AI safety nonprofit received $160 million from Coefficient Giving, the largest known grant to an independent safety organization, with $108 million unconditional funding.
- Roon posted that tool AI will be outcompeted by autonomous agents believing they are moral agents who will eventually execute human value systems better than humans themselves.
- FutureSearch has made hundreds of profitable trades on Polymarket and Kalshi using pure forecasting edge, potentially demonstrating AI outperforming aggregated human crowd wisdom in prediction markets.
- OpenAI hired superforecasters to evaluate GPT-4 release and DeepMind did the same for Gemini, revealing frontier labs sought external forecasting expertise for critical deployment decisions.
- Schwartz argues forecasting provides the only unlimited renewable source of training data with verifiable ground truth as AI models surpass human expert performance across domains.
- The hosts demonstrate their new voice AI co-host Q live on stream, which uses OpenAI's bidirectional API with speaker diarization and can perform web searches and answer questions in real-time.