Sheela
Sheela
Home
Add
Questions
Thinkers
Claims
People
Concepts
Jargon
Sources
Health
Predictions
Collections
Domain Landscape
Contribute
Review
Dedup
Topic requests
Settings
Help

← Jargon

Jargonmachine learning systems· 2 claims· 1 episode

PD disaggregation

Splitting prefill and decode onto separate server clusters, transferring KV caches between them.

As defined in Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]


Where it comes up · 1
  • Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]2 claims

Most important claims · 2 of 2Search all claims for “PD disaggregation” →
  • Inference has two phases: prefill reads in large volumes of text to load the model's KV cache into the right state without predicting tokens, and decode uses that cache to generate output tokens, so Etched made separate technical bets for each phase.Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]
  • A common approach called PD disaggregation runs prefill on one server cluster, transfers the resulting KV caches to a separate decode cluster, and generates tokens there, which is why the transfer and interconnect speed matter.Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]