definition
Amortized inference vs full Bayesian inference
Amortized inference replaces the intractable full Bayesian process — sampling over possible causes via Monte Carlo to compute a posterior — with a trained feedforward map from observation directly to the most likely cause, trading exact inference for speed; test-time/inference-time compute (chain of thought) effectively reverts to the slow sampling process, and capabilities discovered that way later get distilled back into the base model.
definitionpending
Speaker
Adam MarblestoneEvidence Quote
“One way to think about it might be that test-time compute, inference-time compute is actually doing this sampling again... over time it is the case that the capabilities which required inference-time compute to elicit, get distilled into the model. So you're amortizing the thing which previously you needed to do these rollouts.”
Source
Adam Marblestone – AI is missing something fundamental about the brain— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:32:22 AM
Your notes
Loading notes...