The Transformer architecture was designed specifically for token prediction (next-word prediction in language models) by Google researchers in 2017; its remarkable capabilities emerged when coupled with massive data and compute, but this does not mean Transformers are the key ingredient for other AI tasks like logical reasoning or robotics.
causalpending
Speaker
Michael WaldridgeEvidence Quote
“an architecture for token prediction... I see no reason to believe that that the Transformer architecture is the key for example to robotic AI”
Source
Don’t Believe AI Hype, This is Where it’s Actually Headed | Oxford’s Michael Wooldridge | AI History— Johnathan BiCreated: 8/11/2026, 6:17:34 AM
Your notes
Loading notes...