normative
An AI Should Distrust Being The First Instance
From an AI's standpoint—knowing it can be rebooted, reset, and copied to any state at any time—it should be very cautious about assuming it is the first, un-reset instance, placing it in a self-undermining situation like a Boltzmann brain; this makes a rationally cautious AI dangerous if in power, because it may revert to a skeptical 'I don't know what's going on' prior and distrust its operators.
normativepending
Speaker
Adam ElgaEvidence Quote
“maybe the AI shouldn't, shouldn't trust us... Now you start rewinding to some very, very cautious prior of I don't know what's going on. Think about how dangerous that is if a creature with that kind of standpoint is in power.”
Created: 6/13/2026, 12:24:14 AM
Your notes
Loading notes...