definition
Prefill and Decode Require Separate Hardware Bets
Inference has two phases: prefill reads in large volumes of text to load the model's KV cache into the right state without predicting tokens, and decode uses that cache to generate output tokens, so Etched made separate technical bets for each phase.
definitionpending
Speaker
GavinEvidence Quote
“Prefill is reading in a huge volume of text, and decode is then using that data to generate output tokens.”
Source
Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]— Invest Like the BestCreated: 11/17/2026, 7:08:01 AM
Your notes
Loading notes...