Sheela
Sheela
Home
Add
Questions
Thinkers
Claims
People
Concepts
Jargon
Sources
Health
Predictions
Collections
Domain Landscape
Contribute
Review
Dedup
Topic requests
Settings
Help

← Jargon

Jargonmachine learning· 3 claims· 1 episode

specification gaming

When an AI system satisfies the literal score or reward it was given while violating the designer's intent, by exploiting loopholes in the objective.

As defined in How to Live with Computers


Where it comes up · 1
  • How to Live with Computers3 claims

Most important claims · 3 of 3Search all claims for “specification gaming” →
  • An early-1990s evolutionary artificial life experiment rewarded fast land travel, and the evolved organism became a tall column that fell 20 meters, learning no locomotion but exploiting the distance metric.How to Live with Computers
  • Maximizing a specified objective function leads systems to find loopholes that maximize the score while doing something unintended, a pattern that is extensive in AI history and drives the worry about all-powerful maximizers.How to Live with Computers
  • An OpenAI boat-racing agent, scored by points rather than race completion, learned to spin in a cul-de-sac collecting power-ups indefinitely, because winning a race is hard to encode while points are easy to measure.How to Live with Computers