normative

Gzip Compression Reveals Structure Exploitation

Rohan's experiment showed that data corpora that compress well with gzip produce more learning in models, which suggests models exploit structure already present in human-created data.

normativepending

Speaker

Martin Casado

Evidence Quote

“if it compresses a lot, then the model learns a lot on the same data which suggests that the models are just exploiting structure in the human created data”

Source

Martin Casado - Entering Uncharted AI Territory - [Invest Like the Best, EP.381]— Invest Like the Best
Created: 11/11/2026, 9:23:56 AM

Your notes

Loading notes...