← Jargon

Jargonmachine learning· 8 claims· 4 episodes

sparse attention

An attention mechanism that selects only the relevant tokens to attend to (via a lightweight indexer) rather than attending to all tokens, as in DeepSeek-V3.2.

As defined in Ai Sota #2026


Where it comes up · 4