multimodal
AI that can work with multiple input/output types—text, images, and soon video—rather than text alone.
As defined in et_905
Other definitions
Describes AI models that can accept and produce multiple types of media — text, images, audio, and video — rather than just one.
Every Essential AI Skill in 25 Minutes (2025)
Where it comes up · 24
- The AI Reset Is Here: Search, Jobs, and Everything Else w/ Anish Acharya, Dave Blundin, Salim Ismail6 claims
- Demis Hassabis — Scaling, superhuman AIs, AlphaZero atop LLMs, AlphaFold5 claims
- Demis Hassabis — Scaling, superhuman AIs, AlphaZero atop LLMs, AlphaFold4 claims
- Bret Taylor - The Agent Era - [Invest Like the Best, EP.386]3 claims
- John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI2 claims
- Mark Zuckerberg — AI will write most Meta code in 18 months2 claims
- Shane Legg (DeepMind Founder) — 2028 AGI, superhuman alignment, new architectures2 claims
- Why AI Is Good for Humans (with Reid Hoffman)2 claims
and 16 more
Most important claims · 10 of 51Search all claims for “multimodal” →
- Text data may run out, but video is the most valuable untapped source, and transformers should be trained from day one on large video datasets rather than adding images as a side feature to text-trained models.Gavin Uberti - Real-Time AI & The Future of AI Hardware - [Invest Like the Best, EP.356]
- While a pure-RL, no-priors AlphaZero-style path to AGI is theoretically possible, the quickest and most plausible path is to use all existing world knowledge from the Web via scalable algorithms like transformers as a prior, then add planning and search on top; the final AGI will likely include large multimodal models but not rely on them alone.Demis Hassabis — Scaling, superhuman AIs, AlphaZero atop LLMs, AlphaFold
- Video and audio data remain far from exhausted, so new modalities could produce capability leaps in human interaction that are as disorienting as image and video generation advances have been.Capital, Compute & AI Scaling
- Streets historically served humans, horses, carts, and eventually cars, but were captured by cars so that humans became at fault if something went wrong, so reclaiming streets for multimodal transportation is a good impulse.Conversations with Tyler - Roman Mars
- When the speaker puts a prism in front of the multimodal chatbot's lens (without the chatbot being aware of the prism) and places an object where the prism bends the light rays, the chatbot points to the location where the light appears to come from rather than the actual object location.Will AI outsmart human intelligence? - with 'Godfather of AI' Geoffrey Hinton
- Robotics is harder because it is a data-poor regime, but treating any input—an action, a word, an image pixel—as just a token (as in Gato) lets large models transfer to robotics, and true multimodality produces cross-modal benefit where understanding video improves language, so general multimodal systems will ultimately be more capable than single-modality ones despite being harder to train initially.Demis Hassabis — Scaling, superhuman AIs, AlphaZero atop LLMs, AlphaFold
- Large language models exhibit unreasonably effective emergent properties including implicit concept formation and grounding without explicit multimodal training, which was surprising to many including scaling hypothesis proponents.Demis Hassabis — Scaling, superhuman AIs, AlphaZero atop LLMs, AlphaFold
- Google's Gemini 2.5 benchmarks are excellent (beating GPT-4o and GPT-4 on mathematics, coding, and multimodal tasks), but OpenAI is winning the revenue race, indicating that benchmark performance does not guarantee market success.The AI Reset Is Here: Search, Jobs, and Everything Else w/ Anish Acharya, Dave Blundin, Salim Ismail
- Multimodal AI systems (processing text, code, video, and images simultaneously) are necessary for building world models and universal digital assistants that can understand context and assist with real-world tasks beyond text generation.Sir Demis Hassabis on The Future of Knowledge | Institute for Advanced Study
- Export controls are working: DeepSeek had to spend their calories and time on impressive low-level infrastructure optimizations that American labs didn't need to, because they are using partially nerfed chips that are the only ones NVIDIA is allowed to sell in China — and as a result their model is text-only while every major model is now multimodal.Mark Zuckerberg — AI will write most Meta code in 18 months