definition
Grokking Is Minimum Description Length
Grokking—the transition from a memorized, overfit input-output mapping to a generalizing solution—is an instance of the minimum description length principle: the program that generalizes best is the shortest, achieved by progressively pruning and compressing the mapping, so it is memorization plus regularization rather than a separate mechanism.
definitionpending
Speaker
François CholletEvidence Quote
“Grokking is a very, very old phenomenon. We've been observing it for decades. It's basically an instance of the minimum description length principle.”
Source
Francois Chollet — Why the biggest AI models can't solve simple puzzles— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:26:16 AM
Your notes
Loading notes...