The attention weights from softmax are multiplied by the value vector to produce an output vector, where higher softmax scores preserve the value of words the model has learned to prioritize as important, while lower scores drown out irrelevant words.
factualpending
Speaker
Unidentified Speaker — Illustrated Guide to Transformers Neural Network: A step by… [4Bdc55j80l8]Evidence Quote
“then you take the attention weights and multiply it by your value vector to get an output vector the higher softmax scores will keep the value of the words the model learn is more important the lower scores will drown out their irrelevant words”
Created: 8/13/2026, 9:51:15 AM
Your notes
Loading notes...