normative
Gzip Compression Reveals Structure Exploitation
Rohan's experiment showed that data corpora that compress well with gzip produce more learning in models, which suggests models exploit structure already present in human-created data.
normativepending
Speaker
Martin CasadoEvidence Quote
“if it compresses a lot, then the model learns a lot on the same data which suggests that the models are just exploiting structure in the human created data”
Source
Martin Casado - Entering Uncharted AI Territory - [Invest Like the Best, EP.381]— Invest Like the BestCreated: 11/11/2026, 9:23:56 AM
Your notes
Loading notes...