AI & Tech
AI Scientist Discovers 19 New Things But 70 Percent Were Actually False
Cognitive Revolution
AI in the AM — Week 1 Highlights (June 2026)
What was said
"After a few days it came back and it said, I've discovered 19 new things. They said yeah it's probably at least incrementally novel. And then we convinced me to go through and spend days looking at thousands of lines of code and it went down to like 30% of the discoveries were probably real."
Peter Jansen from Allen Institute tested an AI scientist system that claimed 19 discoveries from 50 research ideas. While initial paper review suggested 70-80% validity, deep code review revealed only 30% were real, with some papers analyzing random number generators that commented 'insert rest of neural network code here.'
Verification status
Attributed speaker claim
Egleze verified who said this and where. The underlying claim has not been independently established by Egleze.
Source and context
Egleze verified who said this and where. The underlying claim has not been independently established by Egleze.
- Programme
- Cognitive Revolution
- Episode
- AI in the AM — Week 1 Highlights (June 2026)
- Clip time
- 50:56
- Original source
- Watch the source segment →
From this episode
Cognitive Revolution