foundation models
Large general-purpose AI models trained on broad data that serve as the base layer for downstream applications.
As defined in cwt_chris-dixon
Other definitions
Large-scale machine learning models trained on vast amounts of diverse data that can be adapted or fine-tuned for many different downstream tasks (as opposed to models trained for a single specific task). Examples include GPT, Claude, Gemini.
Why Bezos Silenced The Post | EP 107Large-scale AI models (like GPT-5, Gemini) trained on massive amounts of diverse data that serve as a foundation for various downstream applications; each generation represents exponential increases in training cost and capability
Eric Schmidt and Craig Mundie with David Rubenstein: Genesis
Where it comes up · 26
- Sir Demis Hassabis on The Future of Knowledge | Institute for Advanced Study6 claims
- Eric Schmidt and Craig Mundie with David Rubenstein: Genesis3 claims
- The future of intelligence | Demis Hassabis (Co-founder and CEO of DeepMind)3 claims
- You need to learn AI in 2024! (And here is your roadmap)3 claims
- The AI Reset Is Here: Search, Jobs, and Everything Else w/ Anish Acharya, Dave Blundin, Salim Ismail2 claims
- AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference2 claims
- Shane Legg (DeepMind Founder) — 2028 AGI, superhuman alignment, new architectures2 claims
- Amjad Masad - The Future of Software Creation - [Invest Like the Best, EP.310]2 claims
and 18 more
Most important claims · 10 of 44Search all claims for “foundation models” →
- AlphaEvolve is a promising hybrid system that uses LLMs to propose candidate solutions and evolutionary computing on top to find novel regions of the search space; combining foundation models with search/reasoning algorithms (evolutionary methods, Monte Carlo tree search) is a fertile direction because the model captures known dynamics while search pushes beyond them toward genuine novelty.Demis Hassabis #2
- Foundation models for AI are becoming so good that certain tasks will be automated, but the bigger opportunity is to reimagine customer experiences (like how the internet reimagined travel booking and search), not just to automate existing workflows.Sequoia VC Reveals: The One Trait That Built Airbnb, DoorDash & Zappos
- Generative applications (built on Foundation models like GPT-3, Dolly, Stable Diffusion) represent a new wave of AI applications distinct from intelligent applications; there is a genuine philosophical debate about whether humans should remain in the loop as these technologies develop.AI: Intelligent, Conscious, or Merely Evil? Peter Thiel | COSM 2022
- Following the AIXI framework, if you have a fantastically good sequence predictor approximating Solomonoff induction, then building a powerful general AGI from it is just one more step—adding search and a reinforcement signal—so foundation models that are excellent sequence predictors compressing the world have already solved much of the problem.Shane Legg (DeepMind Founder) — 2028 AGI, superhuman alignment, new architectures
- Traditional evolutionary computing of the 1990s and early 2000s, despite promising results, could never evolve genuinely new emergent properties—you only ever got a subset of the properties you put in—but combining these methods with modern foundation models may overcome that limitation, mirroring how natural evolution clearly did generate new capabilities from bacteria onward.Demis Hassabis #2
- The next generation foundation models (GPT-5, Gemini 2, Gemini 3) will have training costs of approximately $1 billion each just for the base model, and largest data centers contain 200,000 chips, such that training costs will rapidly consume the entire $500 billion Stargate budget.Eric Schmidt and Craig Mundie with David Rubenstein: Genesis
- Current language models only mimic and blend their training data; truly creative problem solving that steps beyond training requires searching through spaces of possibilities to find hidden gems—as AlphaGo's Move 37 came not from human game data but from search identifying an unlikely-but-plausible move and discovering it was excellent—so we won't see systems exceeding their training data until powerful search is added.Shane Legg (DeepMind Founder) — 2028 AGI, superhuman alignment, new architectures
- Foundation models are large pre-trained neural networks trained with massive resources by big tech companies that can be deployed on specific subtasks with minimal expertise by leveraging APIs or frameworks, democratizing access to capabilities that previously required massive infrastructure.You need to learn AI in 2024! (And here is your roadmap)
- Multimodal AI systems (processing text, code, video, and images simultaneously) are necessary for building world models and universal digital assistants that can understand context and assist with real-world tasks beyond text generation.Sir Demis Hassabis on The Future of Knowledge | Institute for Advanced Study
- Over the last 25 years AI has helped individual scientists—increasing their mobility across ideas, speed, rate of promotion, and decreasing coauthor counts—but for science as a whole it currently narrows the scope of ideas discussed, because these are big-data-seeking models that gravitate to problems with abundant existing data and away from origins questions (origin of space, time, language) that predate the institutions producing data.304 | James Evans on Innovation, Consolidation, and the Science of Science