representations
The internal encoded form in which a model stores information; good representations focus on what matters for a task.
As defined in Fully autonomous robots are much closer than you think – Sergey Levine
Where it comes up · 1
Most important claims · 5 of 5Search all claims for “representations” →
- The 'good news' for robotics is that a robot trying to accomplish a task has a built-in focusing mechanism that structures perception toward what matters, mirroring how humans exhibit tunnel vision and ignore goal-irrelevant information, which makes embodied models better at absorbing auxiliary data sources than passive video prediction.Fully autonomous robots are much closer than you think – Sergey Levine
- Because a robot's sensory stream is extremely temporally correlated, the marginal information from each new observation is far less than the full observation, so past observations and their changes can be represented much more concisely than storing images independently, easing the context/compute burden.Fully autonomous robots are much closer than you think – Sergey Levine
- The thinking needed per control time step may be surprisingly low because, like animals, planning happens in advance (neural correlates of movement appear before the movement) setting initial conditions that then unroll, with reactive feedback occurring at a more basic level of abstraction rather than re-planning each step.Fully autonomous robots are much closer than you think – Sergey Levine
- Predicting video has not produced deep world-understanding the way predicting text has, because a visual scene contains unbounded detail (clouds, water molecules, pedestrians) with no inherent prioritization, whereas text is already abstracted into the bits humans care about, providing good representations focused on what matters.Fully autonomous robots are much closer than you think – Sergey Levine
- The key to overcoming context limits is representing context in the right form for the goal, varying by modality, since humans hold some context symbolically (a shopping checklist as words) and other context spatially or visually (navigating a route), and multimodality has far more potential than image-plus-text alone.Fully autonomous robots are much closer than you think – Sergey Levine