factual

Scaling has more dimensions than compute and data

Scaling curves have more dimensions than the original compute, parameters, and data — notably inference compute and context length — and for long context with cheap inference, training an SSM may be worth a 10x worse training-scaling property because the inference budget for long context windows matters most.

factualpending

Speaker

Aman Sanger

Evidence Quote

“maybe the thing you want to train, is some kind of SSM. Because they’re much, much cheaper and faster at super, super long context. And even if, maybe it was 10 X more scaling properties during training... it’s worth it.”

Source

Cursor Team— Lex Fridman Podcast
Created: 6/13/2026, 3:31:02 AM

Your notes

Loading notes...