causal

Prefill Is Memory-Capacity Bound, Decode Is Bandwidth Bound

Disaggregating prefill from decode opens new chip design trade-offs: prefill, which ingests the prompt and tracks state, is bound by memory capacity, while decode, which generates new tokens, is bound by memory bandwidth.

causalpending

Speaker

Gavin Baker

Evidence Quote

“The disaggregation of prefill and inference really has opened the aperture for making these different trade-offs”

Source

Gavin Baker - Watts and Wafers - [Invest Like the Best, EP.473]— Invest Like the Best
Created: 11/17/2026, 11:20:41 AM

Your notes

Loading notes...