multimodality
A model's ability to process and combine multiple types of input/output such as text, images, and video.
As defined in Mark Zuckerberg — Llama 3, $10B models, Caesar Augustus, & 1 GW datacenters
Other definitions
The ability of an AI system to process and integrate multiple types of input (text, images, audio, video, etc.) and produce outputs across multiple modalities; Liang views it as a useful component but not essential to intelligence itself
DeepSeek's Invisible Founder Just Talked for 4 Hours. It Leaked.
Where it comes up · 4
- Mark Zuckerberg — Llama 3, $10B models, Caesar Augustus, & 1 GW datacenters2 claims
- Anthropic CEO Dario Amodei Talks Scaling Laws, AI Arms Races, and Radical Abundance1 claim
- DeepSeek's Invisible Founder Just Talked for 4 Hours. It Leaked.1 claim
- Ep17. Welcome Jensen Huang | BG2 w/ Bill Gurley & Brad Gerstner1 claim
Most important claims · 5 of 5Search all claims for “multimodality” →
- Pre-training scaling laws are now being supplemented by post-training scaling, multimodality scaling, synthetic data generation, and inference-time scaling, with inference-time scaling now scaling up to achieve intelligence through reasoning, simulation, and reflection rather than just model size.Ep17. Welcome Jensen Huang | BG2 w/ Bill Gurley & Brad Gerstner
- There is no single threshold of intelligence; AGI is not one thing but the progressive addition of different capabilities (multimodality, reasoning, memory, emotional understanding), so AI will surpass humans at most tasks gradually rather than at a single moment.Mark Zuckerberg — Llama 3, $10B models, Caesar Augustus, & 1 GW datacenters
- DeepSeek is deliberately text-first and not pursuing multimodality as a core research area because multimodality is a useful component tool (like search) but is not intelligence itself and does not contribute directly to AGI.DeepSeek's Invisible Founder Just Talked for 4 Hours. It Leaked.
- Emotional understanding should be treated as its own AI modality—distinct from generic video or image—because so much of the human brain is dedicated to understanding people, expressions, and emotions; few in the industry focus on it.Mark Zuckerberg — Llama 3, $10B models, Caesar Augustus, & 1 GW datacenters
- Product layer differentiation (how models are deployed, presented, and integrated into applications like Anthropic's Artifacts feature) creates meaningful competitive advantage separate from the model layer itself, and economics of selling applications on top of models differs significantly from simply providing model access via API.Anthropic CEO Dario Amodei Talks Scaling Laws, AI Arms Races, and Radical Abundance