causal

Multimodal Tokens Enable Robotics Transfer

Robotics is harder because it is a data-poor regime, but treating any input—an action, a word, an image pixel—as just a token (as in Gato) lets large models transfer to robotics, and true multimodality produces cross-modal benefit where understanding video improves language, so general multimodal systems will ultimately be more capable than single-modality ones despite being harder to train initially.

causalpending

Speaker

Demis Hassabis

Evidence Quote

“The token could be an action, it could be a word, it could be part of an image, a pixel, or whatever it is. That’s what I think true multimodality is.”

Source

Demis Hassabis — Scaling, superhuman AIs, AlphaZero atop LLMs, AlphaFold— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:35:13 AM

Your notes

Loading notes...