factual
Models excel at recall but struggle with sparse abstraction
Transformers can repeat exact text in their context window, a capability shaped by training data, but they lack the sparse, gist-based compression humans use, so they struggle to reason over effectively unbounded context.
factualpending
Speaker
Dylan PatelEvidence Quote
“Now models are amazing. But what they really suck at is having infinite context.”
Created: 11/16/2026, 4:01:45 AM
Your notes
Loading notes...