Transformers introduced multi-headed attention where each query vector learns multiple attention maps for each key vector and produces multiple context vectors instead of just one, allowing the model to simultaneously attend to different types of relationships.

factualpending

Speaker

Unidentified Speaker — 10 years of NLP history explained in 50 concepts | From Wor… [uocYQH0cWTs]

Evidence Quote

“they introduced the concept of multi-headed attention where each query Vector learns multiple attention maps for each of the key vectors and produces multiple context vectors instead of just one”

Source

10 years of NLP history explained in 50 concepts | From Word2Vec, RNNs to GPT— Neural Breakdown with AVB
Created: 8/11/2026, 1:00:15 AM

Your notes

Loading notes...