In the decoder's second multi-headed attention layer, the encoder's outputs serve as queries and keys while the first multi-headed attention layer's outputs serve as values, allowing the decoder to decide which encoder inputs are relevant to focus on.
factualpending
Speaker
Unidentified Speaker — Illustrated Guide to Transformers Neural Network: A step by… [4Bdc55j80l8]Evidence Quote
“for this layer the encoders output are the queries in the keys in the first multi-headed attention layer outputs are the values this process matches the encoders input to the decoders input allowing the decoder to decide which encoder input is relevant to put focus on”
Created: 8/13/2026, 9:51:15 AM
Your notes
Loading notes...