factual
Models now run at INT4, FP4, even 1-bit precision
Model precision has dropped far beyond what supercomputing experts of 20 years ago would have believed possible: inference now uses INT4 or FP4, and some people quantize models down to two bits or even a single bit (with a sign bit per group), a trend enabled by hardware-algorithm co-design that improves throughput-to-cost ratios.
factualpending
Speaker
Jeff DeanEvidence Quote
“some people are quantizing models to two bits or one bit, and I think that's a trend that definitely”
Created: 6/13/2026, 3:30:41 AM
Your notes
Loading notes...