definition

Prefill-Decode Disaggregation Moves KV Caches Across Clusters

A common approach called PD disaggregation runs prefill on one server cluster, transfers the resulting KV caches to a separate decode cluster, and generates tokens there, which is why the transfer and interconnect speed matter.

definitionpending

Speaker

Gavin

Evidence Quote

“You'll have one cluster of servers running these prefills.”

Source

Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]— Invest Like the Best
Created: 11/17/2026, 7:08:01 AM

Your notes

Loading notes...