factual
Most alignment difficulty is in the strawberry task
Almost all the difficulty of alignment is contained in aligning an AI on a seemingly mundane task—'make two strawberries identical down to the cellular but not molecular level without destroying the universe'—because even this forces the AI to invent biotechnology, and it is more convenient for it to build sub-superintelligences, prevent its own shutdown, and build fortresses to drive success probability toward one; the deeper problem is that we have no method to get a sufficiently advanced AI to do anything at all reliably.
factualpending
Speaker
Eliezer YudkowskyEvidence Quote
“there's already this whole list of ways that it is more convenient to build the technology for these strawberries if you build your own super intelligences in the environment then you prevent yourself from being shut down or you build giant fortresses around the strawberries”
Created: 6/14/2026, 2:07:58 AM
Your notes
Loading notes...