factual
Blue banana test: verifying outcomes we can't understand
We can get rich empirical feedback on AI safety by setting verifiable challenges—e.g. giving an AI control of an air-gapped computer's keyboard and asking it to root the environment to make a blue banana appear—because we can see whether it succeeded even if we'd never have detected the particular exploit, letting us test whether our interpretability and adversarial methods catch an AI using its best efforts.
factualpending
Speaker
Carl ShulmanEvidence Quote
“This can give us a rich empirical feedback where we're able to identify things that are even an AI using its best efforts to get past our interpretability methods, using its best efforts to get past our adversarial examples.”
Source
Carl Shulman (Pt 2) — AI Takeover, bio & cyber attacks, detecting deception, & humanity's far future— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:35:43 AM
Your notes
Loading notes...