The Transformer architecture was designed specifically for token prediction (next-word prediction in language models) by Google researchers in 2017; its remarkable capabilities emerged when coupled with massive data and compute, but this does not mean Transformers are the key ingredient for other AI tasks like logical reasoning or robotics.

causalpending

Speaker

Michael Waldridge

Evidence Quote

“an architecture for token prediction... I see no reason to believe that that the Transformer architecture is the key for example to robotic AI”

Source

Don’t Believe AI Hype, This is Where it’s Actually Headed | Oxford’s Michael Wooldridge | AI History— Johnathan Bi
Created: 8/11/2026, 6:17:34 AM

Your notes

Loading notes...