causal
Transformer dominance is a self-fulfilling hardware prophecy
Because current GPUs and TPUs are built for very high arithmetic intensity and transformers run well on them, all research concentrates on transformers, creating a self-fulfilling prophecy: a theoretically superior architecture that only achieves half the usable FLOPs on existing chips is worthless even with a 30% compute-efficiency win, because it runs half as fast on the hardware.
causalpending
Speaker
Dylan PatelEvidence Quote
“if it's a 30% compute efficiency win, it's half as fast on the chip.”
Source
@Asianometry & Dylan Patel — How the semiconductor industry actually works— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:26:32 AM
Your notes
Loading notes...