misuse
The risk that a properly functioning AI is used by bad actors to cause harm, as distinct from misalignment.
As defined in Dario Amodei (Anthropic CEO) — The hidden pattern behind every AI breakthrough
Where it comes up · 3
- Anthropic CEO Dario Amodei Talks Scaling Laws, AI Arms Races, and Radical Abundance3 claims
- Dario Amodei (Anthropic CEO) — The hidden pattern behind every AI breakthrough1 claim
- Paul Christiano — Preventing an AI takeover1 claim
Most important claims · 5 of 5Search all claims for “misuse” →
- Misalignment becomes a catastrophic issue before AI is sophisticated enough to discover destructive technologies not on our radar, because the competence needed to bring down civilization when broadly deployed (with a billion copies acting) is much lower than the competence needed to advise one person how to do so; thus alignment is the sequenced existential bottleneck, with bioweapons a possible exception.Paul Christiano — Preventing an AI takeover
- AI models pose two distinct safety risk categories: (1) autonomous behavior risks (models acting autonomously without human oversight), which are not yet significant but will become concerning as models become more capable and autonomous; and (2) misuse risks from distributed actors using models for harmful purposes.Anthropic CEO Dario Amodei Talks Scaling Laws, AI Arms Races, and Radical Abundance
- Anthropic's Responsible Scaling Plan involves running safety tests at each new model scale to check for autonomous misbehavior, misuse for biological weapons, cyber attacks, and nuclear information proliferation, implementing checkpoints and safeguards before deploying increasingly capable systems.Anthropic CEO Dario Amodei Talks Scaling Laws, AI Arms Races, and Radical Abundance
- If a model is powerful enough to take over the world and you can control it (solve alignment), then it follows simply that a model following the wishes of a small subset of people could be used by them to take over the world; thus the very premise of misalignment danger means misuse must be worried about with similar levels of consequence — any plan that actually succeeds must solve both.Dario Amodei (Anthropic CEO) — The hidden pattern behind every AI breakthrough
- SB 1047's original version with pre-armed enforcement (government establishing what tests must be run) is problematic because tests for catastrophic AI scenarios are very new and almost none of the relevant catastrophes have occurred, making government-mandated tests potentially counterproductive or politically misused.Anthropic CEO Dario Amodei Talks Scaling Laws, AI Arms Races, and Radical Abundance