Self-attention is a mechanism used to compute attention weights between all pairs of tokens in the input sequence, allowing the model to weigh the importance of different tokens based on their contextual relevance to each other.

definitionpending

Speaker

Unidentified Speaker — 10 years of NLP history explained in 50 concepts | From Wor… [uocYQH0cWTs]

Evidence Quote

“self-attention is a mechanism used to compute attention weights between all pairs of tokens in the input sequence allowing the model to weigh the importance of different tokens based on their contextual relevance to each other”

Source

10 years of NLP history explained in 50 concepts | From Word2Vec, RNNs to GPT— Neural Breakdown with AVB
Created: 8/11/2026, 1:00:15 AM

Your notes

Loading notes...