causal

Transformer attention is a graded production-system conditional

Transformer attention, computed as a query-key match scaling a value vector, is a graded, learnable version of a production-system conditional ('if key matches query to this extent, add value'), and MLP layers implement the same dot-product-based rule structure, so transformers are production-system-like and trainable by gradient descent.

causalpending

Speaker

Chandra Sripada

Evidence Quote

“if key matches query to that extent, add value... Ultimately, what transformers are doing is tons and tons of dot products”

Source

363 | Chandra Sripada on How LLMs and Humans are Cognitive Cousins— Sean Carroll's Mindscape
Created: 11/17/2026, 6:53:48 AM

Your notes

Loading notes...