PD disaggregation
Splitting prefill and decode onto separate server clusters, transferring KV caches between them.
As defined in Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]
Where it comes up · 1
- Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]2 claims
Most important claims · 2 of 2Search all claims for “PD disaggregation” →
- Inference has two phases: prefill reads in large volumes of text to load the model's KV cache into the right state without predicting tokens, and decode uses that cache to generate output tokens, so Etched made separate technical bets for each phase.Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]
- A common approach called PD disaggregation runs prefill on one server cluster, transfers the resulting KV caches to a separate decode cluster, and generates tokens there, which is why the transfer and interconnect speed matter.Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]