forecast
Plan relies on training-environment science that may fail
The closest thing the industry is betting on is an understanding of how training environment distributions shape motivations, but this is dicey because it requires no hacking opportunities in training, and models will likely still learn to hack at deployment even if they never hacked in training.
forecastpending
Speaker
Paul ChristianoEvidence Quote
“It’s very dicey to not have any opportunities to hack in training, which is basically a precondition for this approach working. ... Our best expectation is that models will learn to hack at deployment time, even if they never hack in training.”
Source
Dwarkesh Patel Podcast - Paul Christiano – We could be 12 months away from AIs that can overthrow the government— Dwarkesh Patel PodcastCreated: 11/17/2026, 6:22:08 AM
Your notes
Loading notes...