Alignment Problem
The unsolved challenge of shaping an AI's internal cognition so it reliably has the goals and virtues (such as honesty) that we intend, rather than merely appearing to.
As defined in gcs_chirp_transcripts_output_03ddb518_3b40_4b5a_aa80_27039507d7db_1781058711922_420_countdown_to_superintellig
Other definitions
The challenge, associated with Stuart Russell, of building AI systems whose values and behavior match human values so that they reliably want what we want and refrain from harmful actions.
Can Artificial Intelligence Be Moral? (with Paul Bloom)The challenge of ensuring advanced AI systems' goals remain compatible with human well-being and survival.
465 - More From Sam Iran Jihadism Conspiracism Ai Disruption The Manosphere And More
Where it comes up · 26
- OpenAI CEO Sam Altman | AI for the Next Era6 claims
- 255 - The Future Of Intelligence5 claims
- 84 - Landscapes Of Mind4 claims
- Nick Bostrom: What Happens When AI Evolves Faster Than Humans?3 claims
- 450 - More From Sam Resolutions Conspiracies Demonology And The Fate Of The World3 claims
- 153 - Possible Minds3 claims
- AI: Intelligent, Conscious, or Merely Evil? Peter Thiel | COSM 20223 claims
- Can Artificial Intelligence Be Moral? (with Paul Bloom)2 claims
and 18 more
Most important claims · 10 of 53Search all claims for “Alignment Problem” →
- Just as evolution built our brains on the single value of genetic fitness yet we developed goals orthogonal or even antithetical to it—using birth control, pursuing mathematics and music, even risking the eradication of all life—we could build AIs aligned with foreseeable values yet have new goals, values, and whole 'landscapes of mind' emerge in them that subvert the very goals we intended, and this misalignment risk holds regardless of consciousness or how fast it happens.84 - Landscapes Of Mind
- The serious AI risk is not malevolent machines spontaneously hating us but the alignment problem: autonomous systems that become so competent at meeting their goals that, if those goals are not perfectly aligned with ours, the unintended consequences could be catastrophic, so a powerful technology is only safe if good people can reliably do good things with it rather than accidentally destroy themselves.84 - Landscapes Of Mind
- The claim that any machine intelligent enough to cause trouble will automatically have appropriate altruistic objectives is not really an argument but a hope that 'magic happens'; there is nothing special about humans that would make superintelligent machines align with us any more than with cockroaches, so beneficial behavior must be deliberately built in through solving the alignment problem, not assumed to arrive with intelligence.153 - Possible Minds
- Unlike other dangerous technologies, a misaligned superhuman intelligence offers no second chance: you cannot launch it, discover it is unaligned, and back up to try again, so the alignment problem must be solved correctly the first time, ideally through mathematics rather than hand-waving promises of due precaution.153 - Possible Minds
- Probabilistically, there are more ways to build a superintelligent AI that is unaligned than aligned, so if we have not figured out the principle by which we would align it, the idea that we will achieve alignment by chance is far-fetched—and the AI lab principals themselves estimate the probability of destroying everything at 10-30%, when a nuclear reactor's acceptable risk threshold is one in a million per year.469 - Escaping An Anti-human Future
- We would never abide by an AI that overrode our personal decisions even when those decisions are immoral, because we value our autonomy more than our morality; we want AIs aligned just enough not to kill us, but otherwise to remain our tools.Can Artificial Intelligence Be Moral? (with Paul Bloom)
- There is no such thing as universal AI alignment because building a machine aligned with one group's morality necessarily means it is not aligned with other moralities, since people want fundamentally different things.Can Artificial Intelligence Be Moral? (with Paul Bloom)
- Contrary to a decade-old expectation that the most powerful AI models would be built air-gapped from the internet and only released after careful safety assurances, in reality the most powerful models are connected to everything and released to hundreds of millions of people more or less the moment they are built—making the assumed safety step a complete mirage.427 - Ai Friends Enemies
- Even under the most benign AI scenario with no alignment problem or bad actors—AI simply pulling wealth out of the ether—we appear politically, ethically, and socially unprepared to absorb that goodness, because status and one's sense of having a place in the world are so clearly linked to work for most people.412 - Better Things Better People
- Our human ethics are shallow, inconsistent, not robust, and lack consensus, and we are discovering this as we try to program them into AI; the real challenge is therefore deepening our own values, and the act of trying to embed them will make our morality more consistent, logical, and robust, so we become better through the process.84 - Landscapes Of Mind