Look-ahead masking prevents the decoder from attending to future tokens by adding a mask matrix before calculating softmax and after scaling scores, filling the top right triangle with negative infinity so that softmax converts these to zero attention scores for future tokens.
factualpending
Speaker
Unidentified Speaker — Illustrated Guide to Transformers Neural Network: A step by… [4Bdc55j80l8]Evidence Quote
“the mask is a matrix that's the same size as the attention scores filled with values of materials and negative infinity x' when you add the mask to the scale attention scores you get a matrix of scores with the top right triangle filled with negative infinity x'”
Created: 8/13/2026, 9:51:15 AM
Your notes
Loading notes...