causal

Resisting gradient descent requires specific value conditions

Whether an AI resists having its values modified by training depends on fairly specific conditions: if its values are consequentialist (caring about world-states) and it anticipates that preserving its current values better produces those states, then it is unsurprising that it would prefer not to be modified—but humans are comfortable being changed by learning rather than by direct neural intervention, so the analogy is imperfect.

causalpending

Speaker

Joe Carlsmith

Evidence Quote

“especially not by learning and growing but by gradient descent directly intervening on your neurons.”

Source

Joe Carlsmith — Preventing an AI takeover— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:37:01 AM

Your notes

Loading notes...