factual

Models excel at recall but struggle with sparse abstraction

Transformers can repeat exact text in their context window, a capability shaped by training data, but they lack the sparse, gist-based compression humans use, so they struggle to reason over effectively unbounded context.

factualpending

Speaker

Dylan Patel

Evidence Quote

“Now models are amazing. But what they really suck at is having infinite context.”

Source

Inside the Trillion-Dollar AI Buildout— Invest Like the Best
Created: 11/16/2026, 4:01:45 AM

Your notes

Loading notes...