definition
Transformers contextualize word vectors via attention
In a transformer, input words are turned into high-dimensional vectors (embeddings) that flow through the network; the attention mechanism repeatedly massages these vectors layer by layer so each word's vector incorporates contextual information from the others, so that by the final layer the last vector carries enough context to predict the next word correctly (e.g., 'homework' after 'the dog ate my').
definitionpending
Speaker
Anil AnanthaswamyEvidence Quote
“it has to keep massaging those vectors such that the words start paying attention to each other.”
Created: 6/13/2026, 12:20:04 AM
Your notes
Loading notes...