parameters
The numerical inputs or coefficients of an economic model whose values are often uncertain or estimated.
As defined in Lars Peter Hansen on Risk, Ambiguity, and Measurement
Other definitions
In neural networks, the adjustable weights and biases that the model learns during training; larger numbers of parameters generally correlate with greater model capacity and capability.
What Is an AI Anyway? | Mustafa Suleyman | TED
Where it comes up · 22
- Ilya Sutskever: Deep Learning | Lex Fridman Podcast #946 claims
- The AI Shortcut: How DeepSeek Cracked ChatGPT’s Secrets for Cheap5 claims
- Generative AI in a Nutshell - how to survive and thrive in the age of AI3 claims
- AI: Grappling with a New Kind of Intelligence | World Science Festival3 claims
- What Is ChatGPT Doing? Episode 1: Introduction3 claims
- Dwarkesh Patel Podcast - The data black hole at the center of AI2 claims
- 434 - Can We Survive Ai2 claims
- Gavin Uberti - Real-Time AI & The Future of AI Hardware - [Invest Like the Best, EP.356]2 claims
and 14 more
Most important claims · 10 of 42Search all claims for “parameters” →
- Uncertainty has three distinct components: risk (a fully specified model assigns probabilities to all future events), ambiguity (uncertainty about how much weight or confidence to place across multiple competing models or unknown parameters), and model misspecification (all models are wrong, requiring robustness to forms of misspecification you cannot fully identify).Lars Peter Hansen on Risk, Ambiguity, and Measurement
- Double descent is a phenomenon where test error exhibits a non-monotonic U-shape as model size increases while data is fixed: performance improves as networks grow, then worsens sharply around the interpolation threshold (where model parameters equal training examples), then improves again as model size continues to grow, and this pattern occurs across virtually all practical deep learning systems without early stopping.Ilya Sutskever: Deep Learning | Lex Fridman Podcast #94
- Unlike classical models, where more parameters overfit and generalize worse, transformers do not fall into this trap as much, so adding more money and compute keeps making them better.Gavin Uberti - Real-Time AI & The Future of AI Hardware - [Invest Like the Best, EP.356]
- Large neural networks do not overfit quickly because stochastic gradient descent finds approximately the point with smallest norm in the high-dimensional subspace of solutions achieving zero training error, and this minimum-norm solution is provably insensitive to small random variations in the training data when dimensionality is high.Ilya Sutskever: Deep Learning | Lex Fridman Podcast #94
- LayWM is designed with only 15 million parameters, which is a tiny fraction of modern large language models that have hundreds of billions or over a trillion parameters, and can be trained end-to-end entirely from raw pixels on a single consumer GPU in a matter of hours.LeWorldModel: Stable End-to-End JEPA World Models From Pixels
- The biggest driver of AI progress is scale, moving from GPT-2 at 1.5 billion parameters to GPT-3 at 175 billion to multitrillion-parameter GPT-4, with companies planning roughly 10x more training spend for next-generation models.Gavin Uberti - Real-Time AI & The Future of AI Hardware - [Invest Like the Best, EP.356]
- The model applies distilled knowledge that it has learned from training to its input, which is why sufficient parameters are necessary for proper language modelingWhat Is ChatGPT Doing? Episode 1: Introduction
- There is probably an underlying evolutionary objective function (survive, procreate, ensure children succeed) from which other human drives emerge, but this doesn't answer the question of life's meaning.Ilya Sutskever: Deep Learning | Lex Fridman Podcast #94
- The largest AI models today have several hundred billion parameters, approaching but still below the scale of human brains which have roughly 100 billion neurons with 5,000 connections each, but training models at human-scale connectivity would require tens of thousands of GPU cards that companies like Microsoft, Meta, and Google possess.AI: Grappling with a New Kind of Intelligence | World Science Festival
- Since GPT-3 (2020), simply scaling models—more parameters, layers, and data—has been sufficient to produce breakthroughs, following scaling power laws for next-word prediction; in terms of actual capacities, performance improves nonlinearly, with abilities like solving math problems or commonsense reasoning emerging in transition-like phases past certain sizes, with notable connections to the physics notion of emergence.230 | Raphaël Millière on How Artificial Intelligence Thinks