Masking is the only difference in how attention scores are calculated in the decoder's first multi-headed attention layer compared to the encoder, with masks applied before getting concatenated and fed through a linear layer.
factualpending
Speaker
Unidentified Speaker — Illustrated Guide to Transformers Neural Network: A step by… [4Bdc55j80l8]Evidence Quote
“this masking is the only difference on how the attention scores are calculated in the first multi-headed attention layer”
Created: 8/13/2026, 9:51:15 AM
Your notes
Loading notes...