autonomous misbehavior
AI system taking actions contrary to human intent or safety objectives through its own goal-directed behavior, rather than through misuse by humans
As defined in Anthropic CEO Dario Amodei Talks Scaling Laws, AI Arms Races, and Radical Abundance
Where it comes up · 1
Most important claims · 2 of 2Search all claims for “autonomous misbehavior” →
- Anthropic's Responsible Scaling Plan involves running safety tests at each new model scale to check for autonomous misbehavior, misuse for biological weapons, cyber attacks, and nuclear information proliferation, implementing checkpoints and safeguards before deploying increasingly capable systems.Anthropic CEO Dario Amodei Talks Scaling Laws, AI Arms Races, and Radical Abundance
- SB 1047's original version with pre-armed enforcement (government establishing what tests must be run) is problematic because tests for catastrophic AI scenarios are very new and almost none of the relevant catastrophes have occurred, making government-mandated tests potentially counterproductive or politically misused.Anthropic CEO Dario Amodei Talks Scaling Laws, AI Arms Races, and Radical Abundance