Multi-headed attention involves splitting the query, key, and value vectors into multiple sub-vectors before applying self-attention, where each split undergoes the same self-attention process as a separate 'head' that learns different attention representations.
factualpending
Speaker
Unidentified Speaker — Illustrated Guide to Transformers Neural Network: A step by… [4Bdc55j80l8]Evidence Quote
“to make this a multi-headed attention computation you need to split the query key in value into adding vectors before applying self attention to split vectors that goes through the same self attention process individually each self attention process is called a head”
Created: 8/13/2026, 9:51:15 AM
Your notes
Loading notes...