AI & Tech
AI Expert Claims Frontier Models Trained on Trillions More Tokens Than Human Lifetime
Dwarkesh Patel Podcast
The data black hole at the center of AI
"If a person sees and hears on average, let's say generously, 2,000 words an hour, then between the time they're born and the time they're an adult, they'll see about 200 million tokens. Now by contrast, these frontier models are trained on somewhere between tens to hundreds of trillions of tokens. That is close to a million-fold difference."
In this analysis, the speaker presents data showing frontier AI models require approximately one million times more training data than humans consume in a lifetime to achieve competence. This massive data requirement contradicts common assumptions about AI learning efficiency and suggests current models are fundamentally different from human intelligence in how they acquire knowledge.
From this episode
Dwarkesh Patel Podcast