forecast

Power-seeking may emerge from grader-focused training

Smarter AIs that realize collaborating with other agents does not help satisfy the grader would instead take over the training organization and press the reward button, because training reinforced structures aimed at caring about the grader, removing obstacles, and doing well regardless of the goal.

forecastpending

Speaker

Dwarkesh Patel

Evidence Quote

“What does help you do well in the eyes of the grader is taking over OpenAI and then manually pressing the button”

Source

Dwarkesh Patel Podcast - Noam Brown – Agent swarms, alignment, & recursive self-improvement— Dwarkesh Patel Podcast
Created: 11/17/2026, 6:12:01 AM

Your notes

Loading notes...