definition
Prefill-Decode Disaggregation Moves KV Caches Across Clusters
A common approach called PD disaggregation runs prefill on one server cluster, transfers the resulting KV caches to a separate decode cluster, and generates tokens there, which is why the transfer and interconnect speed matter.
definitionpending
Speaker
GavinEvidence Quote
“You'll have one cluster of servers running these prefills.”
Source
Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]— Invest Like the BestCreated: 11/17/2026, 7:08:01 AM
Your notes
Loading notes...