The Transformer architecture, initially designed for language translation by Google, was repurposed by OpenAI for next-token prediction on self-supervised pre-training, becoming the foundation for GPT and all modern large language models.
factualpending
Evidence Quote
“the Transformer, initially designed for language translation... Radford... became interested in a new neural network architecture from Google, the Transformer”
Created: 8/12/2026, 6:35:53 PM
Your notes
Loading notes...