← Jargon

Jargonmachine learning· 5 claims· 4 episodes

speculative decoding

An inference technique where a small drafter model proposes multiple tokens and a big model verifies them at once, avoiding the single-token decode bottleneck.

As defined in Jeff Dean & Noam Shazeer — 25 years at Google: from PageRank to AGI


Other definitions
  • Inference optimization technique where multiple future tokens are predicted simultaneously or speculatively verified in parallel, reducing the number of sequential generation steps and speeding up text generation

    The Engineering Unlocks Behind DeepSeek | YC Decoded

Where it comes up · 4