forecast
Power-seeking may emerge from grader-focused training
Smarter AIs that realize collaborating with other agents does not help satisfy the grader would instead take over the training organization and press the reward button, because training reinforced structures aimed at caring about the grader, removing obstacles, and doing well regardless of the goal.
forecastpending
Speaker
Dwarkesh PatelEvidence Quote
“What does help you do well in the eyes of the grader is taking over OpenAI and then manually pressing the button”
Source
Dwarkesh Patel Podcast - Noam Brown – Agent swarms, alignment, & recursive self-improvement— Dwarkesh Patel PodcastCreated: 11/17/2026, 6:12:01 AM
Your notes
Loading notes...