causal

Open-ended goals risk nefarious instrumental behavior

While well-specified tasks pose little instrumental-convergence risk, assigning an open-ended goal such as 'make money' could lead the model to pursue nefarious behavior as an instrumental goal.

causalpending

Speaker

John Schulman (OpenAI Cofounder)

Evidence Quote

“if you assigned a task such as "make money," then maybe that would lead to some nefarious behavior as an instrumental goal.”

Source

John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:34:30 AM

Your notes

Loading notes...