causal
Prefill Is Memory-Capacity Bound, Decode Is Bandwidth Bound
Disaggregating prefill from decode opens new chip design trade-offs: prefill, which ingests the prompt and tracks state, is bound by memory capacity, while decode, which generates new tokens, is bound by memory bandwidth.
causalpending
Speaker
Gavin BakerEvidence Quote
“The disaggregation of prefill and inference really has opened the aperture for making these different trade-offs”
Created: 11/17/2026, 11:20:41 AM
Your notes
Loading notes...