reflective equilibrium
A coherentist method of moral inquiry that adjusts judgments at all levels of generality—from concrete cases to abstract principles—until they form a maximally mutually supportive, coherent belief network.
As defined in 107 | Russ Shafer-Landau on the Reality of Morality
Where it comes up · 4
- Joe Carlsmith — Preventing an AI takeover2 claims
- 107 | Russ Shafer-Landau on the Reality of Morality1 claim
- 30 | Derek Leben on Ethics for Robots and Artificial Intelligences1 claim
- 351 | Peter Singer on Maximizing Good for All Sentient Creatures1 claim
Most important claims · 5 of 5Search all claims for “reflective equilibrium” →
- There is no off-the-shelf, pre-normative process of 'reflection' that you can run an agent through to read off its values; in reality there is a whole pattern of empirical facts about what happens when you put an agent through various processes, and you have to take a substantive normative stand on which development processes preserve what you care about versus which take it off the rails—a stand that itself requires endorsing certain meta-processes.Joe Carlsmith — Preventing an AI takeover
- Based on how analytic ethicists actually reason—taking pre-theoretic intuitions and systematizing them into reflective equilibrium—Carlsmith doubts the 'everyone converges' prediction: an agent starting only with intuitions to maximize paperclips has no mechanism by which reflection injects mind-independent moral truth, so a real paperclipper put through reflection would not arrive at rich human morality; we'll get empirical evidence as AIs are trained.Joe Carlsmith — Preventing an AI takeover
- Rather than Rawls's reflective equilibrium, which adjusts theory and intuitions against each other like a scientist fitting a theory to data, Singer is a foundationalist who rejects the idea that moral intuitions constitute solid data; he prefers to get the foundations as clear as possible by taking the most rational point of view and identifying desirable states of consciousness as intrinsically good.351 | Peter Singer on Maximizing Good for All Sentient Creatures
- When a moral theory produces highly counterintuitive or weird suggestions, this can be evidence that the theory is wrong, since for a moral anti-realist our moral intuitions are the starting point and a system wildly in conflict with them may have been built badly—though it could also mean we should discard a flawed intuition.30 | Derek Leben on Ethics for Robots and Artificial Intelligences
- The proper moral epistemology starts with the intuitions about which we are most confident (e.g., genocide is immoral), builds a coherent network of beliefs from them, checks intuitions for distortion by prejudice or ignorance, and maintains a background openness to error; one can never be certain, but that uncertainty does not distinguish moral knowledge from knowledge in any other sphere.107 | Russ Shafer-Landau on the Reality of Morality