← Jargon

Jargonmachine learning· 3 claims· 1 episode

dense attention

Attention in which each token attends to all previous tokens, making memory fetch scale linearly with context length.

As defined in Dwarkesh Patel Podcast - Reiner Pope – The math behind how LLMs are trained and served


Where it comes up · 1