factual
LLMs Generalize Beyond Memorization
Whether LLMs are merely 'haphazardly stitching together' training data or genuinely generating novelty is empirically tractable: research can measure memorization, which is a feature not a bug, but it is provably accurate that there is at least some generalization to out-of-distribution domains—a deflationary form of 'imagination' or creativity.
factualpending
Speaker
Raphael MilliereEvidence Quote
“is the performance merely explainable by this kind of brute force memorization or is it doing something more, which is genuine novelty at generalizing from the training data to new domains”
Created: 6/13/2026, 12:19:51 AM
Your notes
Loading notes...