definition

Misaligned AI defined by planning, not verbal behavior

The AIs worth worrying about have a specific profile—agency, sophisticated world-modeling, situational awareness, and a planning process that actually drives behavior according to internal criteria (its 'values')—and a model's verbal behavior, which gradient descent can clamp to say whatever we want, is weak evidence about those underlying choice-criteria.

definitionpending

Speaker

Joe Carlsmith

Evidence Quote

“We know that we're going to be able to get models to say what we want to hear. That's the magic of gradient descent.”

Source

Joe Carlsmith — Preventing an AI takeover— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:37:01 AM

Your notes

Loading notes...