Sheela
Sheela
Home
Add
Questions
Thinkers
Claims
People
Concepts
Jargon
Sources
Health
Predictions
Collections
Domain Landscape
Contribute
Review
Dedup
Topic requests
Settings
Help

← Jargon

Jargonmachine learning· 2 claims· 1 episode

int4 quantization

A data compression technique that reduces neural network parameters from full floating-point precision to 4-bit integer representation, reducing memory footprint by a factor of ~8-16 while maintaining acceptable accuracy

As defined in This Tiny Engine Runs Impossibly Big AI Models Locally! (colibrì)


Where it comes up · 1
  • This Tiny Engine Runs Impossibly Big AI Models Locally! (colibrì)2 claims

Most important claims · 2 of 2Search all claims for “int4 quantization” →
  • The dense components of GLM 5.2 are only about 17 billion parameters, or under 10 gigabytes at int4 quantization, and remain resident in RAM at all times.This Tiny Engine Runs Impossibly Big AI Models Locally! (colibrì)
  • At int4 quantization, a 744 billion parameter model is still hundreds of gigabytes, which is why large models require data centers full of GPUs and expensive specialized hardware.This Tiny Engine Runs Impossibly Big AI Models Locally! (colibrì)