int4 quantization
A data compression technique that reduces neural network parameters from full floating-point precision to 4-bit integer representation, reducing memory footprint by a factor of ~8-16 while maintaining acceptable accuracy
As defined in This Tiny Engine Runs Impossibly Big AI Models Locally! (colibrì)
Where it comes up · 1
Most important claims · 2 of 2Search all claims for “int4 quantization” →
- The dense components of GLM 5.2 are only about 17 billion parameters, or under 10 gigabytes at int4 quantization, and remain resident in RAM at all times.This Tiny Engine Runs Impossibly Big AI Models Locally! (colibrì)
- At int4 quantization, a 744 billion parameter model is still hundreds of gigabytes, which is why large models require data centers full of GPUs and expensive specialized hardware.This Tiny Engine Runs Impossibly Big AI Models Locally! (colibrì)