causal
Transformer attention is a graded production-system conditional
Transformer attention, computed as a query-key match scaling a value vector, is a graded, learnable version of a production-system conditional ('if key matches query to this extent, add value'), and MLP layers implement the same dot-product-based rule structure, so transformers are production-system-like and trainable by gradient descent.
causalpending
Speaker
Chandra SripadaEvidence Quote
“if key matches query to that extent, add value... Ultimately, what transformers are doing is tons and tons of dot products”
Created: 11/17/2026, 6:53:48 AM
Your notes
Loading notes...