Predictive AI
Artificial intelligence systems that make consequential decisions about people based on machine learning predictions of future behavior or characteristics; used in hiring, lending, criminal justice, healthcare, and education
As defined in AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference
Other definitions
Machine learning systems used to make consequential decisions about people by predicting what they will do or what will happen to them in the future, based on patterns learned from historical data about similar people
AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference
Where it comes up · 1
- AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference6 claims
Most important claims · 6 of 6Search all claims for “Predictive AI” →
- The fundamental problem with using predictive AI for consequential decisions about people is not primarily a technical accuracy problem but a normative/philosophical problem: when an entity with power exercises that power over an individual through algorithmic prediction, it requires justification beyond technical accuracy.AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference
- Predictive AI systems used in criminal justice have false positive rates that are twice as high for Black defendants as for White defendants, and the overall predictive accuracy is less than 70% (measured by area under the curve), meaning decisions about someone's freedom are being made based on predictions only slightly more accurate than the flip of a coin.AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference
- Predictive AI used in criminal justice (bail/pretrial release decisions) has accuracy (AUC) below 70%, which is only marginally better than random guessing at 50%, yet is used to make consequential decisions about freedom despite this poor predictive power.AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference
- Predictive AI applications are distinguished from benign applications (like an AI tool suggesting when a runner needs more fluids) by their social nature—specifically by involving an entity with power exercising that power over an individual without the individual's meaningful consent or control.AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference
- When evaluating predictive AI accuracy, one must ask about baselines: if humans performing the same task achieved 55% accuracy, then an algorithm achieving 70% is 'pretty damn good', not poorly performing.AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference
- There are an ever-expanding set of domains where predictive AI is being deployed: hiring, lending, criminal justice, healthcare, and education.AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference