causal
Quantization requires whole-picture co-design
Quantization only gets adopted through co-design where someone sees the whole picture: in isolation, algorithm designers reject low precision as risky and irritating, and chip designers defer to those algorithm designers, so without a holistic view of how throughput-to-cost can improve dramatically, the beneficial low-precision path never gets taken.
causalpending
Speaker
Noam ShazeerEvidence Quote
“So you actually need to basically see the whole picture and figure out, "Oh, wait a minute, we can increase our throughput-to-cost ratio by a lot by quantizing."”
Created: 6/13/2026, 3:30:41 AM
Your notes
Loading notes...