value lock-in
A scenario in which a particular set of values or ideology becomes permanently entrenched—e.g., under a world dictatorship—preventing further moral progress indefinitely.
As defined in Will MacAskill on Longtermism and What We Owe the Future
Where it comes up · 8
- 292 - How Much Does The Future Matter3 claims
- 467 - Ea Ai And The End Of Work3 claims
- Fin Moorhouse - Longtermism, Space, & Entrepreneurship2 claims
- AGI is still 30 years away — Ege Erdil & Tamay Besiroglu1 claim
- Sholto Douglas & Trenton Bricken — How LLMs actually think1 claim
- Will MacAskill on Longtermism and What We Owe the Future1 claim
- 228 - Doing Good1 claim
- 207 | William MacAskill on Maximizing Good in the Present and Future1 claim
Most important claims · 10 of 13Search all claims for “value lock-in” →
- Democracy is plausibly a historical bargain in which elites grant workers redistribution and influence because workers are useful and can threaten society by rioting; once human labor is worthless and the police and military are fully automated, that bargain dissolves, and because AI forces can be singly loyal to one individual (unlike human soldiers who can refuse orders), an unprecedented concentration of power becomes possible unless we deliberately choose to align AI to the law and constitution rather than to any one person.467 - Ea Ai And The End Of Work
- Longtermism is the view that the most important aspect of our actions is the impact they have over the very long run, which leads to prioritizing the avoidance of massive catastrophes (nuclear war, man-made pandemics) that could derail continued growth, preventing lock-in of bad value states like global totalitarianism, and spurring continued technological and economic innovation into the future.228 - Doing Good
- While many catastrophic risks (nuclear war, dangerous technology, carbon emissions) push toward greater global centralization to solve coordination failures, excessive centralization carries its own catastrophic risk of stalling moral progress—through value lock-in under a world dictatorship or gradual homogenization where people stop trying to improve morally—and good institutions must thread this needle.Will MacAskill on Longtermism and What We Owe the Future
- There is a danger in alignment succeeding too well: whoever ends up in charge of AI systems—companies or governments—would gain fine-grained control over autonomous minds (e.g. turning up a 'loyalty feature'), raising both the ickiness of controlling an autonomous mind and the value lock-in risk, which is a strong reason to be extremely open and publish the constitutions models are expected to abide by.Sholto Douglas & Trenton Bricken — How LLMs actually think
- Humanity should aim for a 'long reflection'—an extended period of deliberation about values and the good before locking in any irreversible, large-scale decisions about the future—because premature commitment risks permanently foreclosing better outcomes.Fin Moorhouse - Longtermism, Space, & Entrepreneurship
- Coordinating across very long time horizons is difficult because future generations cannot bargain with the present, yet such coordination is crucial to avoid value lock-in and to preserve option value for the future.Fin Moorhouse - Longtermism, Space, & Entrepreneurship
- Rather than aiming at a particular utopia—which is dangerous because we don't know what the good ultimately is—we should aim at 'viatopia,' a way-station society that keeps many options open, enables reflection, debate, and ethical progress, and avoids huge irreversible decisions (like handing control to superintelligence, permanently denying or granting AI rights, or letting power concentrate), analogous to a lost adventurer heading for high ground to survey the terrain before choosing a final destination.467 - Ea Ai And The End Of Work
- AI gives the power to create institutions far more long-lasting than any in history, enabling value lock-in: an AGI-enforced constitution or treaty where all soldiers and police are aligned to a fixed agreement could only be renegotiated by out-playing the AI, while at the individual level AI cults, AI-perfected religious arguments, and self-selected epistemic bubbles let people lock themselves into worldviews—so any stabilizing global governance during the transition must be explicitly temporary.467 - Ea Ai And The End Of Work
- The idea that one actor will use a software-singularity AI to 'lock in' a stable utility function for millions of years looks unlike anything in history; even digital information suffers rapid link rot and digitization has been associated with faster, not slower, cultural change — so the argument that we can preserve utility functions forever is weak.AGI is still 30 years away — Ege Erdil & Tamay Besiroglu
- The lock-in paradox is that to guarantee continued moral progress and avoid premature lock-in of today's likely-flawed values, we will paradoxically need to lock in some meta-values—commitment to free debate, tolerance of opposing moral views, and restrictions on gaining power except by persuasion—much as a liberal constitution constrains any individual's power so that diverse worldviews can coexist and progress can continue.292 - How Much Does The Future Matter