Multi-Headed Attention
Definition
An extension of attention mechanisms where query, key, and value vectors are split into multiple independent sub-spaces, attention is computed separately in each sub-space (a 'head'), and results are concatenated, allowing the model to simultaneously attend to different representation subspaces
Cast within
No topic-region cast yet — this appears once Multi-Headed Attention's compiled claims are aligned into a topic region's argument tree.
Claims mentioning Multi-Headed Attention (13)
Your notes
Loading notes...