Self-attention is a mechanism used to compute attention weights between all pairs of tokens in the input sequence, allowing the model to weigh the importance of different tokens based on their contextual relevance to each other.
definitionpending
Speaker
Unidentified Speaker — 10 years of NLP history explained in 50 concepts | From Wor… [uocYQH0cWTs]Evidence Quote
“self-attention is a mechanism used to compute attention weights between all pairs of tokens in the input sequence allowing the model to weigh the importance of different tokens based on their contextual relevance to each other”
Source
10 years of NLP history explained in 50 concepts | From Word2Vec, RNNs to GPT— Neural Breakdown with AVBCreated: 8/11/2026, 1:00:15 AM
Your notes
Loading notes...