AI & Tech
Data Movement Costs Dominate Logic Operations by Factor of 7 to 1
Dwarkesh Patel Podcast
Reiner Pope – Chip design from the bottom up
"So this circuit I've described here, almost all of the cost, like 7/8 of the cost is in the reading and writing the register file. And only a tiny fraction of the cost is in the logic unit itself."
Pope demonstrated through circuit analysis that in traditional CPU and GPU cores, 87% of the transistor area and energy goes to moving data between registers and compute units, with only 13% performing actual calculations. This fundamental inefficiency was the primary motivation for introducing specialized tensor cores and systolic arrays in modern AI accelerators.
From this episode
Dwarkesh Patel Podcast