Current autoregressive transformers already handle predicting high-dimensional distributions with uncertainty effectively, as evidenced by their ability to predict the next page in a book or generate images, contradicting claims that new architectures are necessary for this capability.

factualpending

Speaker

Ilya Sutskever

Evidence Quote

“I'll give you two examples. One is given one page in a book, predict the next page in a book. There could be so many possible pages that follow. It's a very complicated high-dimensional space and we deal with it just fine.”

Source

The Mastermind Behind GPT-4 and the Future of AI | Ilya Sutskever— Eye on AI
Created: 8/10/2026, 10:48:45 PM

Your notes

Loading notes...