parameter
An internal adjustable weight in a neural network; model size is often described by parameter count (e.g. a 400-billion-parameter model).
As defined in et_982
Other definitions
A learnable variable in a neural network whose values are adjusted during training to optimize performance; more parameters allow larger models to represent more complex functions
LeWorldModel: Stable End-to-End JEPA World Models From Pixels
Where it comes up · 13
- This Tiny Engine Runs Impossibly Big AI Models Locally! (colibrì)8 claims
- Run Local LLMs on Hardware from $50 to $50,000 - We Test and Compare!7 claims
- AI: Grappling with a New Kind of Intelligence | World Science Festival4 claims
- The Future of Knowledge Graphs in a World of Large Language Models4 claims
- LeWorldModel: Stable End-to-End JEPA World Models From Pixels3 claims
- Feed Your OWN Documents to a Local Large Language Model!2 claims
- What the Heck is Graph Engineering?2 claims
- Machine Learning in Investing1 claim
and 5 more
Most important claims · 10 of 36Search all claims for “parameter” →
- LayWM is designed with only 15 million parameters, which is a tiny fraction of modern large language models that have hundreds of billions or over a trillion parameters, and can be trained end-to-end entirely from raw pixels on a single consumer GPU in a matter of hours.LeWorldModel: Stable End-to-End JEPA World Models From Pixels
- GLM 5.2 is a 744 billion parameter language model in the same size class as frontier models running in data centers, but Colibri enables it to run on consumer hardware.This Tiny Engine Runs Impossibly Big AI Models Locally! (colibrì)
- Research on how knowledge is stored in large language model parameters is beginning to understand why models must be so large; Stable Diffusion 1 has only 890 million parameters (200x smaller than GPT-3's 175 billion) because it doesn't need to internalize knowledge of individual entities.The Future of Knowledge Graphs in a World of Large Language Models
- The largest AI models today have several hundred billion parameters, approaching but still below the scale of human brains which have roughly 100 billion neurons with 5,000 connections each, but training models at human-scale connectivity would require tens of thousands of GPU cards that companies like Microsoft, Meta, and Google possess.AI: Grappling with a New Kind of Intelligence | World Science Festival
- Out of 744 billion total parameters in GLM 5.2, only 40 billion parameters are actually active per token, which is roughly 5% of the total model size.This Tiny Engine Runs Impossibly Big AI Models Locally! (colibrì)
- ByteDance is in the early stages of training an ultra-large model with as many as 10 trillion parameters, which could take 3 to 6 months to complete with additional time needed for reinforcement learning after that.What the Heck is Graph Engineering?
- Older models like PLDM attempted to prevent representation collapse by designing loss functions with seven different interacting terms, creating extreme training instability due to the combination lock problem—adjusting one parameter unpredictably shifts all others.LeWorldModel: Stable End-to-End JEPA World Models From Pixels
- Model size has increased exponentially from around 2018 to 2021 and beyond, with the number of parameters growing such that when plotted on a log scale it appears as a straight line, representing sustained exponential growth in AI model capacity.AI: Grappling with a New Kind of Intelligence | World Science Festival
- Neural networks in artificial intelligence work analogously to biological neurons: signals are represented as numbers, networks compare inputs against learned filters or patterns, and parameters representing connection strengths are adjusted during training to make outputs match desired results.AI: Grappling with a New Kind of Intelligence | World Science Festival
- Training and inference compute costs have followed a multiplicative growth pattern: GPT-3 cost $100 million with 175 billion parameters, while GPT-4 cost $400 million (4x) but with nearly 10x the parameters, suggesting costs continue to grow with each subsequent model generation.The Possibilities of AI [Entire Talk] - Sam Altman (OpenAI)