causal
Arithmetic area scales quadratically with bit width
Because multiplier circuit area scales quadratically (p×q) with bit width, halving the precision should more than quadruple — not merely double — the achievable FLOP count, which is the single biggest reason low-precision arithmetic has worked so well for neural nets; Nvidia historically doubled FLOPs per halved precision but only acknowledged the larger speedup (3x FP4 vs FP8) in B300 and beyond, though it arguably should be 4x.
causalpending
Speaker
Reiner PopeEvidence Quote
“This is a really big reason... because of this quadratic scaling, that ratio is actually slightly wrong. You should get an even bigger speedup than you might otherwise think.”
Created: 6/13/2026, 3:33:43 AM
Your notes
Loading notes...