definition

In-Context Learning vs Continual Learning

There are two paths to fast adaptation: continual learning (continuously updating model weights so it adapts to new information rapidly, like an employee who learns from feedback) versus in-context learning (loading a huge context window with extra information each prompt); Lambert is more bullish on simply providing very good context, since updating weights per person is too expensive.

definitionpending

Speaker

Nathan Lambert

Evidence Quote

“I’m personally more bullish on language models being able to just provide very good context.”

Source

Ai Sota #2026— Lex Fridman Podcast
Created: 6/13/2026, 3:24:41 AM

Your notes

Loading notes...