causal
Multimodal Tokens Enable Robotics Transfer
Robotics is harder because it is a data-poor regime, but treating any input—an action, a word, an image pixel—as just a token (as in Gato) lets large models transfer to robotics, and true multimodality produces cross-modal benefit where understanding video improves language, so general multimodal systems will ultimately be more capable than single-modality ones despite being harder to train initially.
causalpending
Speaker
Demis HassabisEvidence Quote
“The token could be an action, it could be a word, it could be part of an image, a pixel, or whatever it is. That’s what I think true multimodality is.”
Source
Demis Hassabis — Scaling, superhuman AIs, AlphaZero atop LLMs, AlphaFold— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:35:13 AM
Your notes
Loading notes...