factual

Pretraining Data Is Like Fossil Fuels

Pretraining data is analogous to fossil fuels: just because it is non-renewable does not mean civilization is on a dead-end track, because you couldn't have transitioned from water wheels in 1800 directly to solar and fusion without a cheap, convenient, plentiful intermediary.

factualpending

Speaker

Dwarkesh Patel

Evidence Quote

“You simply couldn’t have transitioned from the water wheels in 1800 to solar panels and fusion power plants. We had to use this cheap, convenient, plentiful intermediary.”

Source

Some thoughts on the Sutton interview— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:28:01 AM

Your notes

Loading notes...