Generative model
An internal model encoded in the brain of the causes of sensory signals, which supplies the predictions tested against sensory prediction errors.
As defined in 168 | Anil Seth on Emergence, Information, and Consciousness
Other definitions
A machine learning model that learns to generate or reconstruct data (such as pixel values), often by predicting complete outputs from inputs
Metas AI Boss Says He DONE With LLMS...A machine learning model that learns to generate or reconstruct outputs (like pixels or tokens) from inputs; aims to model the full distribution over possible outputs, making predictions in the output space rather than representation space
Yann LeCun: World Models: Enabling the next AI revolution
Where it comes up · 16
- 235 | Andy Clark on the Extended and Predictive Mind7 claims
- 87 | Karl Friston on Brains, Predictions, and Free Energy6 claims
- Yann LeCun: World Models: Enabling the next AI revolution3 claims
- 320 - Constructing Self And World3 claims
- 168 | Anil Seth on Emergence, Information, and Consciousness3 claims
- Eliezer Yudkowsky on the Dangers of AI2 claims
- 113 - Consciousness And The Self2 claims
- 322 - Predicting Reality2 claims
and 8 more
Most important claims · 10 of 37Search all claims for “Generative model” →
- Because only the tiny foveal macula sees in color and detail while peripheral vision lacks color and saccades produce momentary blindness, the vivid, stable, fully-colored visual scene we experience cannot be the raw sensory data but must be the output of the brain's generative model.320 - Constructing Self And World
- Hinton's restricted Boltzmann machines trained on handwriting could generate digits from random noise, morphing a seven into a nine and a one into a two through smooth transitions they had never seen, which Lowin initially took as intelligence before deciding it was a deterministic outcome of model construction.Machine Intelligence and Risk Management
- The key difference between a planning creature and a virus is that some biotic systems install in their generative model the prior belief that they are free-energy-minimizing creatures who will act to minimize future free energy; expressed psychologically, this is simply the capacity to plan or imagine counterfactual futures.87 | Karl Friston on Brains, Predictions, and Free Energy
- Often the correct move for a prediction-driven system is to temporarily increase its own uncertainty so as to better minimize prediction errors over the long run, which gives surprise its value; we artificially curate environments to surprise ourselves, and art and science may largely be the curation of environments that harvest surprises improving our generative models.235 | Andy Clark on the Extended and Predictive Mind
- Perceiving something as a three-dimensional object with a back you cannot see consists in the brain encoding conditional/counterfactual predictions about how sensory data would change if you moved around or interacted with it, implemented via a generative model; this explains why synesthetic colors, which lack such sensorimotor contingencies, do not acquire the phenomenology of objecthood.113 - Consciousness And The Self
- Perceiving something as a volumetric object with a back we cannot see is constituted by the brain encoding conditional or counterfactual predictions about how sensory data would change if we acted on it (moved around it, picked it up); this explains why synesthetic colors, which lack such sensorimotor contingencies, are experienced as additional rather than as properties of objects in the world.264 - Consciousness And Self Rebroadcast
- Rather than naming a precise evolutionary point where systems become planners, Friston invokes the philosophical notion of vagueness (the sorites/heap problem): just as there is no exact number of grains that makes a 'pile,' there is no sharp threshold for when systems acquire the capacity to plan, since even simple systems like thermostats have a minimal notion of the future via rates of change.87 | Karl Friston on Brains, Predictions, and Free Energy
- Predictive brains are tuned to patterns that solved problems before and direct attention to where good information is expected, so we systematically miss outlier and unusual events, as in inattentional blindness (the gorilla experiment) and 'looked-but-didn't-see' accidents where expert drivers miss a cyclist arriving from an unexpected direction even after moving their heads that way.235 | Andy Clark on the Extended and Predictive Mind
- The canonical computation of the brain is attempting to predict the current flow of sensory information; at the basic time scale it is predicting the present, acting as a guessing machine that combines prior experience with current sensory evidence to estimate the true state of the world, which lets us use the past to make better sense of present sensory signals.235 | Andy Clark on the Extended and Predictive Mind
- The brain constructs perception by issuing top-down predictions (from a generative model) about likely sensory stimulations and using them to clean up, discard, and amplify incoming signal; sensory information is swapped for prediction error early in processing, so everything we see, touch, and feel is framed by these predictions rather than read directly off the world.322 - Predicting Reality