definition
Transformer Architecture Explained
A transformer is the architecture behind ChatGPT consisting of many layers of large continuous vectors representing words, concatenated to the model's context length; word vectors are randomly initialized then learned by optimizing the next-word prediction objective, and each word's representation is updated via an attention mechanism that takes a weighted average over neighboring words' representations, all words updating simultaneously, which is why it scales so efficiently.
definitionpending
Speaker
Yejin ChoiEvidence Quote
“each word is enhanced with technical term called attention mechanism... update your own representation as a weighted... average over all the words in your neighborhood”
Created: 6/13/2026, 12:23:11 AM
Your notes
Loading notes...