factual

Context should be represented in varied task-relevant modalities

The key to overcoming context limits is representing context in the right form for the goal, varying by modality, since humans hold some context symbolically (a shopping checklist as words) and other context spatially or visually (navigating a route), and multimodality has far more potential than image-plus-text alone.

factualpending

Speaker

Sergey Levine

Evidence Quote

“there's other things that are much more spatial, almost visual... I'm just thinking, "milk."”

Source

Fully autonomous robots are much closer than you think – Sergey Levine— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:29:18 AM

Your notes

Loading notes...