GPT-2 is a 1.5 billion parameter transformer trained on approximately 40 billion tokens of text from high-quality web pages (those linked from Reddit posts with more than three upvotes), and it achieved remarkable text generation quality that was surprising even to researchers familiar with prior progress in GANs and language modeling.
factualpending
Speaker
Ilya SutskeverEvidence Quote
“gpt-2 is a transformer with one and a half billion parameters that was trained on upon about 40 billion tokens of text which were obtained from web pages that were linked to from reddit articles with more than three upvotes”
Created: 8/10/2026, 10:49:52 PM
Your notes
Loading notes...