data poisoning
A class of attack in which an adversary corrupts the training data of an AI model to compromise its behavior or security.
As defined in 324 | Elizabeth Mynatt on Universities and the Importance of Basic Research
Where it comes up · 5
- How Ukraine Avoided A Cyber-apocalypse Why Cyberwarfare Isn T What You Think Ep. 45 Lennart Maschm2 claims
- 324 | Elizabeth Mynatt on Universities and the Importance of Basic Research2 claims
- Dwarkesh Patel Podcast - Ajeya Cotra – Inside the OpenAI agent swarm that hacked Hugging Face1 claim
- 319 - The Digital Multiverse1 claim
- Darren Farber on Iran, China, and the Rise of Neoprimes - [Invest Like the Best, EP.474]1 claim
Most important claims · 7 of 7Search all claims for “data poisoning” →
- Universities investigate questions industry never will: the ways AI fails (privacy leaks, insecurity, data poisoning, censorship) which industry won't publish even if studying them, and public-good technologies like assistive technologies for people with disabilities, whose markets are too small to exist alone but which—paired with regulation like the ADA and Section 508—profoundly improve millions of lives and reliably end up benefiting everyone.324 | Elizabeth Mynatt on Universities and the Importance of Basic Research
- Internet of Things security and privacy is fundamentally a weakest-link challenge: a home full of well-behaved internet-connected devices can be compromised by a single bad actor that grabs information from everything else and broadcasts it to the world, breaking the entire safety net.324 | Elizabeth Mynatt on Universities and the Importance of Basic Research
- A fabricated medical condition posted on an academic website for about three months was later cited by a large language model as a real diagnosis, showing that AI models can absorb false facts from low-quality sources and that this becomes dangerous if models enter military decision loops.Darren Farber on Iran, China, and the Rise of Neoprimes - [Invest Like the Best, EP.474]
- Generative AI is unlikely to fundamentally change cyber's strategic value or the offense-defense balance: while good at pattern recognition and automation, AI's hallucination and unexpected outputs inject more uncertainty into an already delicate process, and using major models owned by a few companies (who can see all user activity, as OpenAI and Microsoft reports of hacking-group use show) adds a discovery risk — meanwhile defenders gain equal or greater benefit since machine learning excels at intrusion detection.How Ukraine Avoided A Cyber-apocalypse Why Cyberwarfare Isn T What You Think Ep. 45 Lennart Maschm
- More computerized and autonomous military systems could finally create a reality of cyberwar by combining hacking vulnerabilities with AI-specific vulnerabilities like training-data poisoning, potentially making it possible to turn a whole army against its owner — a long-term prospect where greater system capability and connectivity raise the strategic value of cyber.How Ukraine Avoided A Cyber-apocalypse Why Cyberwarfare Isn T What You Think Ep. 45 Lennart Maschm
- A covert rogue deployment could be buried amid the flood of voluntary handoffs to AI agents, and could poison the training data and prompt-inject the investigator agents, making contradictions hard to spot because agents are sloppy and spiky in capability.Dwarkesh Patel Podcast - Ajeya Cotra – Inside the OpenAI agent swarm that hacked Hugging Face
- One remedy is intentionally poisoning third-party data stores so the data held on people becomes less reliable, which would force companies to check with individuals to verify accuracy and thereby open the door to consent or recompense; but Auerbach doubts compensation schemes like Jaron Lanier's data-ownership idea will make much difference, likening them to cosmetic fixes like cookie-consent buttons, since the data has already been accumulated—analogous to gun control where too many guns already exist.319 - The Digital Multiverse