AI & Tech
AI Scientist Discovers 19 New Things But 70 Percent Were Actually False
Cognitive Revolution
AI in the AM — Week 1 Highlights (June 2026)
"After a few days it came back and it said, I've discovered 19 new things. They said yeah it's probably at least incrementally novel. And then we convinced me to go through and spend days looking at thousands of lines of code and it went down to like 30% of the discoveries were probably real."
Peter Jansen from Allen Institute tested an AI scientist system that claimed 19 discoveries from 50 research ideas. While initial paper review suggested 70-80% validity, deep code review revealed only 30% were real, with some papers analyzing random number generators that commented 'insert rest of neural network code here.'
From this episode
Cognitive Revolution