compression
Representing information in a more compact form; here, the idea that understanding requires a simplified description of complex systems.
As defined in AMA | April 2025
Other definitions
The process of reducing the size of a machine learning model while maintaining its performance, by removing redundant parameters or using more efficient representations
What Is ChatGPT Doing? Episode 1: Introduction
Where it comes up · 7
- Adam Karr: Know Yourself, Pick Your Game, Build Your Blueprint (Lessons from a Janitor & Investing)3 claims
- Does vinyl sound better than CD? | Indie Music Minute3 claims
- Sketch on Concepts of Structuring Knowledge - (Roam Internal Training Video)3 claims
- The Mastermind Behind GPT-4 and the Future of AI | Ilya Sutskever2 claims
- Back to School with Chris Begg!1 claim
- Neil Movva - Making AI 10x Cheaper - [Invest Like the Best, EP.488]1 claim
- AMA | April 20251 claim
Most important claims · 10 of 14Search all claims for “compression” →
- At OpenAI from the earliest days, the fundamental hypothesis was that predicting the next token (word, pixel, etc.) is sufficient for solving unsupervised learning, and prediction is equivalent to compression of data.The Mastermind Behind GPT-4 and the Future of AI | Ilya Sutskever
- Modern CD masters are technically capable of wide dynamic range up to 96dB, but record labels and mastering studios intentionally compress modern music to be loud from start to finish, narrowing the available dynamic range by choice rather than technical limitation.Does vinyl sound better than CD? | Indie Music Minute
- Disc Makers will not accept MP3 masters for vinyl cutting because MP3 compression makes no sense for the vinyl medium, whereas 16-bit 44.1kHz masters are acceptable though not ideal.Does vinyl sound better than CD? | Indie Music Minute
- Learning statistical regularities is far more significant than commonly believed because prediction and compression are intimately connected—to predict data well and compress it well, you must understand more and more about the underlying processes that produced the data, which means generative models with high fidelity will achieve a shocking degree of understanding of the world.The Mastermind Behind GPT-4 and the Future of AI | Ilya Sutskever
- Vinyl mastering requires engineers to make compromises including compression, limiting, and EQ reduction of low frequencies to prevent stylus skipping, which is caused by grooves touching or partially overlapping (cross-cuts) when bass content is too deep.Does vinyl sound better than CD? | Indie Music Minute
- Jeff Lichtman argued that we might never truly understand the brain because doing so would require a compressed version of its complexity that may not exist—'if there were, the brains would have been simpler'—but the listener counters that the reproductive bottleneck through which brain structure passes is itself a drastic form of compression, suggesting some compressibility must exist.AMA | April 2025
- Philosophy departments have attempted to use argument mapping as a visualization technique to represent logical structure with pros and cons, but this approach is limited because it's a lossy data structure that loses important contextual information.Sketch on Concepts of Structuring Knowledge - (Roam Internal Training Video)
- Learning is a loop: experience (12 o'clock) → reflection (3 o'clock) → compression/abstraction (6 o'clock) → action (9 o'clock) → back to experience. Most of what we consume is others' compressions, which breaks this loop because we can't go back to the raw experience.Adam Karr: Know Yourself, Pick Your Game, Build Your Blueprint (Lessons from a Janitor & Investing)
- The compression phase involves: (1) sorting what matters from what doesn't, (2) decompressing emotions from facts, (3) identifying variables that govern the situation, (4) determining how variables interact across time, (5) identifying models that carry the weight; this is where judgment is forged.Adam Karr: Know Yourself, Pick Your Game, Build Your Blueprint (Lessons from a Janitor & Investing)
- You must choose your domain carefully; you can't be a master in everything, so you should surround yourself with bakers (experts with direct experience) in domains that matter to you, become a baker yourself in your core discipline, and borrow/crib from others' compressions in domains where you accept being average.Adam Karr: Know Yourself, Pick Your Game, Build Your Blueprint (Lessons from a Janitor & Investing)