In the decoder's second multi-headed attention layer, the encoder's outputs serve as queries and keys while the first multi-headed attention layer's outputs serve as values, allowing the decoder to decide which encoder inputs are relevant to focus on.

factualpending

Speaker

Unidentified Speaker — Illustrated Guide to Transformers Neural Network: A step by… [4Bdc55j80l8]

Evidence Quote

“for this layer the encoders output are the queries in the keys in the first multi-headed attention layer outputs are the values this process matches the encoders input to the decoders input allowing the decoder to decide which encoder input is relevant to put focus on”

Source

Illustrated Guide to Transformers Neural Network: A step by step explanation— The AI Hacker
Created: 8/13/2026, 9:51:15 AM

Your notes

Loading notes...