factual
Utility functions are a lossy approximation of agency
Treating real-world human agents as having utility functions is at best a very lossy approximation that becomes increasingly misleading as agent intelligence rises; agency and planning genuinely produce outcomes (a person searches and ends up with a house and dog; civilization gets a man on the moon) without requiring any consistent utility function, and this picture is more real than utility functions.
factualpending
Speaker
Joe CarlsmithEvidence Quote
“It's very janky...thinking of humans as having utility functions is at best a very lossy approximation.”
Created: 6/13/2026, 3:37:01 AM
Your notes
Loading notes...