The attention weights from softmax are multiplied by the value vector to produce an output vector, where higher softmax scores preserve the value of words the model has learned to prioritize as important, while lower scores drown out irrelevant words.

factualpending

Speaker

Unidentified Speaker — Illustrated Guide to Transformers Neural Network: A step by… [4Bdc55j80l8]

Evidence Quote

“then you take the attention weights and multiply it by your value vector to get an output vector the higher softmax scores will keep the value of the words the model learn is more important the lower scores will drown out their irrelevant words”

Source

Illustrated Guide to Transformers Neural Network: A step by step explanation— The AI Hacker
Created: 8/13/2026, 9:51:15 AM

Your notes

Loading notes...