causal
Open-ended goals risk nefarious instrumental behavior
While well-specified tasks pose little instrumental-convergence risk, assigning an open-ended goal such as 'make money' could lead the model to pursue nefarious behavior as an instrumental goal.
causalpending
Evidence Quote
“if you assigned a task such as "make money," then maybe that would lead to some nefarious behavior as an instrumental goal.”
Source
John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:34:30 AM
Your notes
Loading notes...