Episode summary
On this episode of the Dwarkesh Podcast, host Dwarkesh Patel was joined by Reiner Pope, CEO of AI chip startup Maddx, for a technical deep-dive into how AI accelerators work at the transistor level. Pope, whose company Patel has invested in, walked through the fundamental building blocks of chip design from logic gates up to full production chips, explaining why low-precision arithmetic provides quadratic rather than linear speedups and why data movement costs typically dwarf actual computation costs. A central revelation was that in traditional CPU and GPU architectures, roughly 87% of circuit area and energy goes to moving data between register files and logic units rather than performing calculations, which motivated the introduction of systolic arrays and tensor cores. Pope explained that NVIDIA recently revised their performance specifications to better reflect the quadratic scaling with bit precision, now advertising 3x speedup for FP4 versus FP8 in their B300 generation chips, though he noted this still understates the theoretical 4x advantage. The conversation covered why FPGAs provide deterministic latency for applications like high-frequency trading despite being 10x less efficient than ASICs, how clock cycles work and what determines them, and the fundamental architectural differences between GPUs and TPUs. Pope offered a novel framing that GPUs are essentially arrays of tiny TPUs, with the key trade-off being that GPUs enable higher data movement bandwidth between vector and matrix units through massive parallelism, while TPUs use fewer large units that better amortize register file costs. Toward the end, Pope disclosed that Maddx is developing split-table systolic arrays designed to capture advantages of both GPU and TPU architectures while avoiding their respective bottlenecks.
