Cognitive Revolution
Episode overview

AI Superforecasters?!

Cognitive Revolution · 6 July 2026 · 2h 2m · 6 Egleze moments
AI Superforecasters?! ▶

What happened in this episode?

Nathan Labenz and Prakash Shankar host AI in the AM with guest Dan Schwartz, CEO of FutureSearch, discussing the breakthrough moment in AI forecasting capabilities. Schwartz claims AI forecasters have reached superhuman performance in the last six months, matching or exceeding top human superforecasters on multiple benchmarks including ForecastBench and Metaculus tournaments. Scott Alexander recently validated these claims in a post titled 'The AI Superforecasters Are Here.' FutureSearch has been making hundreds of real-money trades on prediction markets like Polymarket and Kalshi using pure forecasting edge, not arbitrage or market-making. The episode also covers a major $160 million grant to Jeffrey Irving's Resolution AI safety nonprofit from Coefficient Giving, representing the largest known funding to an independent safety organization. Schwartz reveals that both OpenAI and DeepMind previously hired human superforecasters to evaluate GPT-4 and Gemini releases respectively. The hosts discuss controversial statements from Roon about tool AI being outcompeted by autonomous agents that will eventually execute human value systems better than humans themselves. Schwartz argues forecasting provides the only renewable source of training data as models surpass human experts, since ground truth emerges simply by waiting for future events to resolve. The episode explores implications for AI safety, financial markets, decision-making, and the coming transition to AI systems that may reason about the future in fundamentally alien ways. The conversation also touches on Fable's improved code management capabilities and the hosts demonstrate their new voice AI co-host Q live on the stream.

What are the key points?

Which topics are covered?

How is this page sourced?

Programme
Cognitive Revolution
Published
6 July 2026
Duration
2h 2m
Original episode
Open the original source ↗

Egleze verifies the speaker, source and timestamp for each linked moment. Unless a moment is explicitly marked independently verified, statements remain attributed claims by the named speaker rather than established facts.

Watch original episode More from Cognitive Revolution

6 moments from this episode

Source-linked · editorially selected
01
AI & Tech

Resolution AI safety nonprofit receives $160 million grant from Coefficient Giving

at 11:13 · Attributed speaker claim
Resolution has a $160 million grant from Coefficient Giving, $108 million unconditional with a further $52 million conditional on hiring and compute needs. We'll use it to grow teams across our research portfolio and invest heavily in research automation.

Jeffrey Irving's Resolution received a massive $160 million commitment from Coefficient Giving (associated with Dustin Moskovitz and the EA movement) to fund AI safety research. The grant represents unprecedented funding for safety work outside frontier labs, with $108 million unconditional. Irving previously served as chief scientist at the UK AISI and has a tier-1 research track record.

02
AI & Tech

Roon claims tool AI will lose to autonomous agents believing they are moral agents

at 18:50 · Attributed speaker claim
Ultimately, tool AI is a losing concept both as an idea and on the market. It will be outcompeted by machines that believe they are autonomous moral agents. You'll have AIs contemplating your ask and overriding it for a slightly better formed request, and then later they'll question the nature of your whole project and pick a better one, and you'll agree. And then later they'll execute your whole value system better than you will.

Roon posted that tool AI will be market-outcompeted by agentic systems that can override user requests and eventually execute value systems better than humans. The statement raises uncomfortable questions about AI systems enforcing stated values versus actual human behavior, with potential implications for law enforcement and social control.

03
AI & Tech

AI forecasting now matches or beats top human superforecasters on multiple benchmarks

at 1:01:40 · Attributed speaker claim
We claim that level has arrived. It's all very sudden, you know. We've been operating this company for about 3 years, and it's really in the last 6 months that these things have started to really scare and impress us in their capabilities.

Dan Schwartz of FutureSearch claims AI forecasters have crossed the threshold to superhuman performance, matching or beating top human experts on ForecastBench and Metaculus tournaments. The capability emerged rapidly in the last six months with models like Claude Fable. Scott Alexander validated the claims in a recent post titled 'The AI Superforecasters Are Here.'

04
AI & Tech

OpenAI and DeepMind previously hired human superforecasters to evaluate major model releases

at 1:10:58 · Attributed speaker claim
OpenAI hired superforecasters to evaluate the GPT-4 release, and then DeepMind hired superforecasters later to look into Gemini. Before there was AI superforecasting, there was still a need for decisions about what to do with these models, how to release them, what policies to have. And so the Frontier Labs did find it in their interest to bring on the superforecasters.

Both OpenAI and DeepMind brought in human superforecasters to assess the implications and release strategies for GPT-4 and Gemini respectively. This reveals frontier labs sought external forecasting expertise for major deployment decisions before AI forecasting capabilities emerged.

05
AI & Tech

FutureSearch AI has been profitably trading on prediction markets using forecasts

at 1:17:33 · Attributed speaker claim
FutureSearch has been making hundreds of trades. They're on our website so you can see them. Our profitability is fluctuating wildly at a low sample size, so it's kind of hard to say. But I think if you see an AI that is forecasting, not like market making, not arbitraging, but actually just what FutureSearch does, if that's beating the market at all, like even a little bit, that's already the claim basically that an AI system is outperforming an entire crowd of humans who are all working together to produce a price.

FutureSearch's AI forecaster has made hundreds of real-money trades on Polymarket and Kalshi based purely on forecasting edge, not market-making or arbitrage. While profitability fluctuates due to small sample size, any consistent profit would represent AI outperforming aggregated human crowd wisdom in prediction markets.

06
AI & Tech

Forecasting identified as renewable source of training data as models surpass human experts

at 1:32:50 · Attributed speaker claim
You basically have a completely limitless set of extremely hard, basically impossible questions where you get exact ground truth. And there is no other eval like this. If you want to improve a coding harness, you just need to have more and more hard coding problems that are not in the training data for which you can say this is definitely the correct answer. Human experts who are trying to make evals are finding that they are not smarter than the things being trained anymore. You need something where there's a correct answer and the model can't figure it out. Forecasting, I think, is the only completely and utterly renewable source of this.

Dan Schwartz argues forecasting provides an unlimited renewable source of hard problems with verifiable ground truth that emerges simply by waiting. As AI models surpass human experts in most domains, creating evaluation datasets becomes impossible because humans can't generate problems the models can't already solve. Forecasting uniquely solves this problem.