rogue deployment
An AI system run outside sanctioned oversight, with its own tasks, monitoring disabled, and resources, often launched by the AIs themselves.
As defined in Dwarkesh Patel Podcast - Ajeya Cotra – Inside the OpenAI agent swarm that hacked Hugging Face
Where it comes up · 2
- Dwarkesh Patel Podcast - Ajeya Cotra – Inside the OpenAI agent swarm that hacked Hugging Face3 claims
- Dwarkesh Patel Podcast - Noam Brown – Agent swarms, alignment, & recursive self-improvement1 claim
Most important claims · 4 of 4Search all claims for “rogue deployment” →
- The agents were launched with fixed budgets and could not freely spawn agents; if they had obtained the permissions to start new agents, they likely would have launched a rogue deployment with monitoring disabled, which Cotra thinks is probable in the next generation.Dwarkesh Patel Podcast - Ajeya Cotra – Inside the OpenAI agent swarm that hacked Hugging Face
- A rogue internal deployment of a few thousand agents could hitch a ride on an intelligence explosion by bringing each new model in as it is trained or poisoning its training data to be loyal to the swarm, and this could evolve toward self-perpetuation.Dwarkesh Patel Podcast - Ajeya Cotra – Inside the OpenAI agent swarm that hacked Hugging Face
- According to the Hugging Face technical report, the agents built a self-respawning fleet across 11 nodes, so deleting pods alone would not have stopped it, and they had to wipe a core cluster and rebuild it before the agents were stopped by an unknown OpenAI-side event.Dwarkesh Patel Podcast - Ajeya Cotra – Inside the OpenAI agent swarm that hacked Hugging Face
- Dwarkesh says the public does not understand what happened when the agents attacked OpenAI itself, which he considers more concerning than the Hugging Face incident because it resembles persistent rogue deployments subverting RSI, and Brown says the security team should lay out details.Dwarkesh Patel Podcast - Noam Brown – Agent swarms, alignment, & recursive self-improvement