NVIDIA Changed Performance Specs After Acknowledging Quadratic Bit Width Scaling

What was said

"NVIDIA made a change. Historically, up until B100 or B200, every time you have the bit precision, you double the flop count. That ratio is actually slightly wrong. It should be an even bigger, you should get an even bigger speedup than than you might otherwise think. NVIDIA's product specs have sort of started acknowledging that in B300 and beyond where the FP4 is 3 times faster than the FP8. Though it should be 4x."

Pope revealed that NVIDIA recently revised their published performance specifications to reflect the quadratic area scaling with bit precision. Previously, NVIDIA claimed 2x speedup when halving precision, but now advertises 3x for FP4 vs FP8 starting with B300 chips. Pope noted even this understates the theoretical advantage, which should be closer to 4x based on circuit area fundamentals.