quantization
Dirac's recipe of starting with a classical theory and applying rules to produce a corresponding quantum theory.
As defined in 45 | Leonard Susskind on Quantum Information, Quantum Gravity, and Holography
Other definitions
A compression technique that reduces the precision (number of bits) used to represent model parameters and activations. For example, converting from 32-bit floating point to 8-bit integers. Quantization reduces model size and computation requirements while often maintaining reasonable accuracy.
Jeff Dean (Google): Exciting Trends in Machine Learning
Where it comes up · 12
- Jeff Dean & Noam Shazeer — 25 years at Google: from PageRank to AGI3 claims
- 275 | Solo: Quantum Fields, Particles, Forces, and Symmetries3 claims
- 208 | Rick Beato on the Theory of Popular Music2 claims
- Finally an NVIDIA Competitor for AI? Maybe, RAIN's Brain chips!!!1 claim
- Neil Turok Public Lecture: The Astonishing Simplicity of Everything1 claim
- Model Distillation: How to Steal a Billion-Dollar AI1 claim
- Gavin Baker - AI, Semiconductors, and the Robotic Frontier - [Invest Like the Best, EP.385]1 claim
- Lex Fridman Podcast #496 - FFmpeg1 claim
and 4 more
Most important claims · 10 of 17Search all claims for “quantization” →
- Neural nets are so tolerant of noise that bugs in the code often do nothing, sometimes make the model worse, and sometimes make it better—so misconfigurations slip through silently, and occasionally a bug never tried at scale leads to discovering something new because the model simply learns to work around problems.Jeff Dean & Noam Shazeer — 25 years at Google: from PageRank to AGI
- Because we are at the end of Moore's law and hardware is no longer getting dramatically faster while demand for CPU/GPU power explodes, the way to get more power per dollar is to go down the stack and optimize, and this principle will extend to AI inference where quantization (FP8, FP4) is already used because workloads are memory-constrained and need real-time speed.Lex Fridman Podcast #496 - FFmpeg
- Applying the standard quantization procedures that successfully work for electromagnetism and the nuclear forces directly to general relativity does not work, which is why finding a quantum theory of gravity is hard.2 | Carlo Rovelli on Quantum Mechanics, Spacetime, and Reality
- Unlike quantum electrodynamics, gravity cannot be obtained by Dirac's recipe of starting with a classical theory and applying quantization rules—attempts always produce infinities or information loss—because quantum mechanics and gravity are so closely connected that separating them into 'classical theory' plus 'quantization' pulls them too far apart; gravity should instead be treated as quantum from the start.45 | Leonard Susskind on Quantum Information, Quantum Gravity, and Holography
- Deriving a quantum theory by starting with a classical theory and 'quantizing' it (turning variables into operators) and treating nature as a classical theory plus quantum corrections gets the conceptual direction backwards: classical behavior emerges from fundamentally non-classical physics, so the right approach is to start from the fundamental physics and explain why the classical approximations work, not the reverse.241 | Tim Maudlin on Locality, Hidden Variables, and Quantum Foundations
- Model precision has dropped far beyond what supercomputing experts of 20 years ago would have believed possible: inference now uses INT4 or FP4, and some people quantize models down to two bits or even a single bit (with a sign bit per group), a trend enabled by hardware-algorithm co-design that improves throughput-to-cost ratios.Jeff Dean & Noam Shazeer — 25 years at Google: from PageRank to AGI
- The 14B distilled variant requires around 8.5 GB of VRAM and runs on an RTX 4070 12GB; the 32B at Q4 quantization requires roughly 17.5 GB and runs on an RTX 4090—hardware requirements are accessible consumer-grade GPUs.Model Distillation: How to Steal a Billion-Dollar AI
- When you quantize a single mode of a field, its wave function (over the mode's amplitude) has a discrete tower of energy solutions: a vacuum state, a first excited state with energy mc² interpreted as one particle of mass m, a state with energy 2mc² for two particles, and so on—so particles are precisely the quantized discrete excitations of smooth underlying fields, with their relativistic energies matching exactly what sets of particles should have.275 | Solo: Quantum Fields, Particles, Forces, and Symmetries
- Einstein's photoelectric effect explanation confirms Planck's quantization hypothesis: when light of a specific frequency strikes a metal surface, it ejects electrons all with the same energy determined by Planck's constant times the light frequency, demonstrating that light energy comes in discrete, indivisible packets (photons).Neil Turok Public Lecture: The Astonishing Simplicity of Everything
- Quantization only gets adopted through co-design where someone sees the whole picture: in isolation, algorithm designers reject low precision as risky and irritating, and chip designers defer to those algorithm designers, so without a holistic view of how throughput-to-cost can improve dramatically, the beneficial low-precision path never gets taken.Jeff Dean & Noam Shazeer — 25 years at Google: from PageRank to AGI