definition

Amortized inference vs full Bayesian inference

Amortized inference replaces the intractable full Bayesian process — sampling over possible causes via Monte Carlo to compute a posterior — with a trained feedforward map from observation directly to the most likely cause, trading exact inference for speed; test-time/inference-time compute (chain of thought) effectively reverts to the slow sampling process, and capabilities discovered that way later get distilled back into the base model.

definitionpending

Speaker

Adam Marblestone

Evidence Quote

“One way to think about it might be that test-time compute, inference-time compute is actually doing this sampling again... over time it is the case that the capabilities which required inference-time compute to elicit, get distilled into the model. So you're amortizing the thing which previously you needed to do these rollouts.”

Source

Adam Marblestone – AI is missing something fundamental about the brain— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:32:22 AM

Your notes

Loading notes...