Constitutional AI
Anthropic's approach of giving a model a set of guiding principles (a 'constitution') that shape its behavior and values.
As defined in Lex Fridman Podcast #491 - OpenClaw
Other definitions
An alignment method (associated with Anthropic) that uses a set of principles and AI feedback to shape model behavior.
Shane Legg (DeepMind Founder) — 2028 AGI, superhuman alignment, new architectures
Where it comes up · 6
- Dario Amodei (Anthropic CEO) — The hidden pattern behind every AI breakthrough4 claims
- Dario Amodei2 claims
- The AI Tsunami is Here & Society Isn't Ready | Dario Amodei x Nikhil Kamath | People by WTF1 claim
- Carl Shulman (Pt 2) — AI Takeover, bio & cyber attacks, detecting deception, & humanity's far future1 claim
- Shane Legg (DeepMind Founder) — 2028 AGI, superhuman alignment, new architectures1 claim
- Lex Fridman Podcast #491 - OpenClaw1 claim
Most important claims · 10 of 10Search all claims for “Constitutional AI” →
- Sampling from a foundation model is like System 1 blurting out the first thing, and RLHF/Constitutional AI only poke a high-dimensional distribution at many points so they aren't robust; instead alignment requires a System 2 that reasons step by step, enumerates options, uses a good world model to predict consequences, and evaluates each option ethically before acting—mirroring how a person handles a hard ethical decision.Shane Legg (DeepMind Founder) — 2028 AGI, superhuman alignment, new architectures
- If safety schemes rely on one AI policing others, a takeover becomes possible only when all AIs in the system both desire takeover and find an opportunity to coordinate and act simultaneously, so that no AI interrupts another's steps toward takeover.Carl Shulman (Pt 2) — AI Takeover, bio & cyber attacks, detecting deception, & humanity's far future
- Alignment is better understood as a statistical, practice-improvable skill — like learning to juggle five balls — than as a binary problem to be solved once like the Riemann hypothesis; we are not aligned-by-default nor doomed-by-default, and over two to three years we expect to incrementally 'eat that probability mass' of ways things can go wrong by getting better at diagnosing and controlling models, with interpretability as the test set.Dario Amodei (Anthropic CEO) — The hidden pattern behind every AI breakthrough
- Steinberger created a private soul.md file, inspired by Anthropic's constitution, that infuses the agent with personality and core values, with the agent allowed to modify its own soul (on the condition he knows), having written its own soul file and the onboarding templates that give each user's agent its character.Lex Fridman Podcast #491 - OpenClaw
- Meaningful safety research requires being at or beyond the frontier because methods like debate and amplification need models coherent enough for a human to judge their debates, and empirical learning — discovering ten new ideas and ten new failure modes by trying methods in practice — is the bulk of the work; even interpretability benefits from powerful models auto-interpreting weaker ones, so scaling and safety are 'two snakes coiled together.'Dario Amodei (Anthropic CEO) — The hidden pattern behind every AI breakthrough
- The constitution does not tell the model exactly how to behave; because training interacts with human data and pre-existing biases, principles act as nudges whose strength can be tuned — e.g., a 'never ever' principle may shift a behavior from 40% to 80% rather than producing literal absolutes.Dario Amodei
- Constitutional AI samples two model responses, has the AI itself judge which better follows a human-readable document of principles (a constitution), and feeds that judgment into a preference model — a triangle of AI, preference model, and AI improvement — reducing the need for human RLHF while keeping principles human-interpretable.Dario Amodei
- Anthropic's positions on safety, interpretability, alignment, and constitutional AI are not marketing-driven claims because warning about model dangers and advocating for regulation that constrains Anthropic commercially are 'not effective marketing strategies.'The AI Tsunami is Here & Society Isn't Ready | Dario Amodei x Nikhil Kamath | People by WTF
- There should not be one constitution everyone uses; a model's constitution should be very simple, containing only basic facts everyone would agree on (drawn initially from sources like the UN Declaration of Human Rights and Apple's Terms of Service), with many ways to customize and append constitutions, and Amodei is uncomfortable with the idea of a single AI constitution running the world as fanciful given how societies and politics actually work.Dario Amodei (Anthropic CEO) — The hidden pattern behind every AI breakthrough
- Terms like drives, goals, and thoughts are inadequate for describing what happens when we apply RLHF or Constitutional AI to a model; we have very little idea what we're talking about and are essentially making up words, which is why mechanistic interpretability — being able to say a specific circuit turns on or off — is needed to describe model psychology honestly.Dario Amodei (Anthropic CEO) — The hidden pattern behind every AI breakthrough