Multi-Headed Attention

Definition

An extension of attention mechanisms where query, key, and value vectors are split into multiple independent sub-spaces, attention is computed separately in each sub-space (a 'head'), and results are concatenated, allowing the model to simultaneously attend to different representation subspaces

Cast within

No topic-region cast yet — this appears once Multi-Headed Attention's compiled claims are aligned into a topic region's argument tree.

Claims mentioning Multi-Headed Attention (13)

Your notes

Loading notes...