Naively showing a multimodal LLM a picture and asking what it would do fails, because the model does not know it is a robot, what the robot looks like, or what is going on, so connecting LLM knowledge to robot control is a real technical challenge.

causalpending

Speaker

Sergey Levine

Evidence Quote

“it doesn't know that you are a robot, this is what you look like, this is what's going on”

Source

World's Top Researcher on AI, LLMs, and Robot Intelligence— Invest Like The Best
Created: 11/11/2026, 5:04:53 AM

Your notes

Loading notes...