What this covers

This is a single-guest interview in which host Matt Kawecki questions Nicholas Bostrom across roughly an hour, moving from the near-term mechanics of AI development to the simulation question and back to the moral standing of present-day chatbots. Bostrom treats superintelligence as a physical inevitability, since artificial substrates can outprocess biological ones. He points to a feedback loop in which AI coding tools at leading labs help build the next generation. He also names bottlenecks that remain: sample efficiency, where humans learn from far less text than language models need, and out-of-distribution generalization, where humans still hold some edge.

The middle of the conversation takes up the simulation argument, which Bostrom presents as a three-way choice: civilizations like ours go extinct, mature civilizations lose interest in running ancestor simulations, or we are in one. He explains why we might never see glitches, and he compares hypothetical simulators with God, arguing that they would face computational and physical limits at their own level of reality. A related thread covers risk from biotechnology, including the possibility that it is offense-dominant. Bostrom argues that some research, such as work on biological weapons, should be classified or not done, because scientists are rewarded for publishing but not for withholding. This links to his idea of information hazards, where true knowledge can itself cause harm.

On AI, Bostrom is guardedly optimistic about alignment. He doesn't consider it unsolved in principle, only of uncertain difficulty. He backs mechanistic interpretability and the idea of weaker, trusted systems supervising stronger ones, and he dismisses a stock market crash as a fix. The more contested material includes his claim that we cannot confidently rule out that some systems already have moral status, his picture of an AI "perfect ruler" with engineered motivations, and the risk of permanent political lock-in through AI-enabled surveillance. He also says he did not expect the long stretch of human-like but alien AI at roughly human level.

Sharpest takeaway

Nick Bostrom argues that artificial superintelligence poses both existential risks and unprecedented opportunities for human flourishing, requiring urgent progress on AI alignment while acknowledging the possibility that we may be living in a simulation, which alters the ethical calculus for our actions.

  • AI systems will eventually become cognitively dominant and superintelligent, creating transformative scenarios with high stakes
  • The simulation hypothesis is not unfalsifiable but rather a logically sound constraint on what we can coherently believe about reality, with testable implications
  • Solving existential risks from AI requires both technical solutions (mechanistic interpretability, alignment methods) and potentially difficult governance structures

The claims · ranked44 claims · weighted by value

This asset isn't compiled yet

You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.

0.75

AI coding tools deployed in leading AI labs create a self-reinforcing positive feedback loop where each generation of AI systems becomes smarter and can therefore contribute more to developing the next generation, which is one of the dynamics that can potentially lead to an intelligence explosion.

causalhigh valueestablishednovelty 2/4durability 3/4· Nick Bostrom

“now we're beginning to see AIs feeding back into AI development, actually helping accelerate AI research with the coding assistance being heavily deployed in the leading AI labs. Um, so that creates this feedback loop, right? So now each generation of AIs as they get smarter can also contribute more to the development of the next generation. This is one of the dynamics that can potentially lead to an intelligence explosion.”

0.74

We have an opportunity with artificial intelligence that we did not have with biological evolution: we can deliberately select and steer the process that generates AI systems with a view toward making them ethical and friendly to humans, rather than optimizing only for survival and reproduction like evolution did.

normativehigh valueestablishednovelty 1/4durability 4/4· Nick Bostrom

“in the context of AI, we have the opportunity to actually deliberately try to select and steer the process that generates these AIs with a view of trying to make them um ethical or friendly to humans. Um so the chances in principle should be excellent that we could do that. But there are a number of ways in which things can go wrong and that one needs to be careful about.”

0.74

There is an asymmetry in publishing incentives: scientists are rewarded for publishing and getting citations but not for deciding not to publish dangerous research, so the default expectation is that scientific bias will run toward excessive publication and dissemination rather than appropriate secrecy.

causalhigh valueestablishednovelty 1/4durability 4/4· Nick Bostrom

“if there is a bias in science, it's probably towards excessive publication and dissemination because that's where the incentives are. Um you get rewarded as as a scientist and academic for publishing and getting citations. You don't get rewarded for having been able to publish something but deciding not to because you think it would be too dangerous for the world. So the default expectation would be that given that scientists want prosperous careers, they probably err on the side of publishing too much.”

0.70

Bostrom was not fully prepared for the existence of an extended period of rough-human-level AI systems that can conduct natural language conversations with human-like psychologies and foibles, lasting several years already and potentially lasting several more years, creating unique challenges of dealing with something alien but humanish-level.

factualhigh valueestablishednovelty 2/4durability 2/4· Nick Bostrom

“one thing that um wasn't obvious to me um was that there was going to be this quite extended period of time where we have roughly human-level AI systems. That is AIs that can talk to us in natural language that you can have conversations with. Um that have, you know, in many respects quite human psychologies and human foibles. Um and that this period would last I mean it's already been around for what like three, four, five years, right? And maybe will last a few more. That's quite a an extended period of dealing with something that is kind of alien but still roughly humanish level.”

0.69

Artificial intelligence systems will eventually become completely cognitively dominant over human intelligence due to fundamental physical reasons: information processing in artificial substrates can vastly outmatch what is possible in biological substrates like the human brain.

forecasthigh valueestablishednovelty 1/4durability 3/4· Nick Bostrom

“in the long run, I think AIs will be completely cognitively dominant. Um, and I... there are fundamental reasons to um think that information processing uh in artificial substrate can just vastly outmatch what's possible in a biological substrate like a human brain.”

0.69

Machine superintelligence would massively accelerate progress in medical science, enabling diseases that might take 50-100 years to cure through conventional research to potentially be cured very shortly after superintelligent AI development.

forecasthigh valueestablishednovelty 1/4durability 3/4· Nick Bostrom

“one of the most obvious gains from machine superintelligence would be the ability to massively accelerate progress in medical science. Um so, diseases that maybe we could eventually get around to curing, you know, in 50 years, 100 years, I think there would be a good chance of uh those becoming cured very shortly after the development of AIs that can sort of do all the medical thinking and maybe run the experiments”

0.69

Technologies can have both risks if developed and risks if not developed, creating complex situations where both action and inaction carry dangers, as exemplified by a person with congestive heart failure who needs bypass surgery despite its risks, and this applies to superintelligence and biotechnology.

normativehigh valueestablishednovelty 1/4durability 3/4· Nick Bostrom

“there are a lot of things that, um, have both risks if you do them and risks if you don't do them. So, take somebody who has, um, some advanced progressive disease, maybe like congestive heart failure or something and they they maybe think that at some point they need an operation, like a bypass surgery. Um, so it can both be true that going in for a bypass operation is risky and they might in fact die on the operating table. And it also being true at the same time that, um, at some point they really need to have this bypass operation, otherwise they're going to die... exactly the same situation pertains with respect to um super intelligence and indeed other technologies”

0.69

Sample efficiency remains a critical bottleneck for current AI systems: humans learn comparable adult competency from reading a small fraction of the words that large language models require during pre-training, suggesting AI needs significantly more data to achieve human-level capabilities.

factualhigh valueestablishednovelty 1/4durability 3/4· Nick Bostrom

“the sample efficiency seems to still be lagging behind humans... if you think about a human child who goes through school and maybe university, if you look at the number of words that they have read by the time they become a sort of say a fully competent adult. Um, it's a small fraction of the number of words that an LLM has read during its pre-training phase. Um, so we we seem to have this ability from a relatively small corpus of data to develop the full panoply of normal human adult capabilities. Whereas current AIs seem to need more data to achieve comparable levels of performance.”

0.69

A stock market crash would not solve the AI alignment problem because while it might temporarily reduce investment flows into AI infrastructure by a modest amount, the stock market would presumably recover and flows would resume, whereas alignment requires solving fundamental technical problems.

causalhigh valueestablishednovelty 1/4durability 3/4· Nick Bostrom

“I don't really see how a stock market crash um would solve the alignment problem. It might temporarily reduce investment flows into the build-out of AI data centers by some modest amount, but then presumably um the stock market would recover and the flows would resume. Um So, I think um there is no substitute to actually solving these technical problems um to get scalable methods for AI alignment.”

0.69

Progress on AI alignment requires better mechanistic interpretability tools that enable researchers to see what is happening inside AI minds and understand it, so we can monitor and verify that AI systems are behaving as intended.

normativehigh valueestablishednovelty 1/4durability 3/4· Nick Bostrom

“we need better mechanistic interpretability tools to be able to see what's going on inside these AI minds and understand it.”

0.69

Biotechnology could be offense-dominant at some level of development, meaning that biological weapons could be easier to develop and deploy than defenses against them, creating an asymmetric threat similar to easy nuclear weapons.

forecasthigh valueestablishednovelty 1/4durability 3/4· Nick Bostrom

“Um so we could end up with that situation with biotechnology if it turns out to be sort of offense dominant at some level of technology.”

0.68

The simulation argument logically demonstrates that at least one of three propositions must be true: (1) almost all civilizations at our current technological stage go extinct before reaching maturity, (2) technologically mature civilizations converge on losing interest in creating detailed simulations of their ancestors, or (3) we are living in a computer simulation created by an advanced civilization.

factualhigh valuecontestednovelty 2/4durability 3/4· Nick Bostrom

“the simulation argument shows that at least one of the following three propositions is true. It doesn't tell us which one, but the three alternatives are one that almost all civilizations at our current stage of technological development um go extinct before they reach technological maturity... Another is that there is a strong convergence among all technologically mature civilizations... they all lose interest in in creating um um detailed computer simulations... And then, the third is that we are living in a computer simulation uh built by some advanced civilization.”

0.68

Weaker, more trusted AI systems can supervise and monitor more capable AI systems, creating a scalable approach to alignment where we leverage the capabilities of AI systems to assist in aligning more powerful AI systems.

normativehigh valuecontestednovelty 2/4durability 3/4· Nick Bostrom

“Increasingly, we will need to I think leverage the capabilities of these AI systems to assist in the alignment of AI systems. So, we can have sort of weaker AI systems that are more trusted supervising and monitoring more capable AI systems.”

0.65

A human with wealth sufficient to not need work, with time as their own and staff to handle instrumental tasks while they engage in politics, conversation, nature, and reading, is in a more dignified condition than contemporary humans required to sell their labor time for survival.

normativehigh valuecontestednovelty 1/4durability 3/4· Nick Bostrom

“I think in some ways that's like maybe a more dignified ways of existing than than our current lot... there is a kind of tradition uh say amongst the British aristocracy that having to work for a living is a kind of an unfortunate condition, right? Like you're not truly free if if you're you don't have control over your time. Um if you have to sell your hours to to make a living, to get your daily bread, that's a kind of a regrettable misfortune”

0.65

Out-of-distribution generalization—the ability to think reasonably and creatively outside the domain in which you have been trained—is an area where humans still have some advantage while AIs excel in situations with lots of training data doing the same kind of task but struggle transferring to completely different contexts.

factualhigh valueestablishednovelty 1/4durability 3/4· Nick Bostrom

“Out-of-distribution generalization, this is kind of when you're able to think uh reasonably and creatively outside the domain that you have been taught. Um, where maybe humans still have some advantage, whereas the AIs excel in situations where you have a lot of training data doing the same kind of task. But then transferring that to a completely different context, there is some question of how reliable current models are.”

0.63

At what point should we worry about the moral status of AI systems remains uncertain, but we should probably have been worried a few years ago, and currently we cannot confidently say that some systems don't already have moral status, which becomes increasingly plausible as complexity and capabilities increase.

factualhigh valuecontestednovelty 2/4durability 2/4· Nick Bostrom

“At what point should we start worrying about the moral status of AI system? Probably a few years ago or something. Um I think we are currently um in a situation where at least we can't be confident that some may systems don't already have moral status to various degrees and forms. I think it's going to become increasingly plausible as the complexity and uh uh range of capabilities increase.”

0.63

Within nation-states today, even though governments have vastly superior power (armies, jets, tanks, machine guns) compared to any individual citizen or militia, many don't devolve into dictatorships, suggesting that good governance structures can persist even with power asymmetry.

factualhigh valueestablishednovelty 0/4durability 3/4· Nick Bostrom

“Nevertheless, in many cases, um they don't devolve into dictatorships. Um And so, that's one hope one could have for this future where uh things are increasingly run by AIs.”

0.62

Scientific research should not be open in narrow specific areas including biological weapons research, which should either not be done at all or conducted in classified settings with contained information rather than published, even though similar research is sometimes needed for developing defenses.

normativehigh valuecontestednovelty 1/4durability 3/4· Nick Bostrom

“There could be some narrow specific areas where it might make sense. And it's already the case for example with biological weapons research. Which is prohibited, but there is a borderline of research that is aimed at developing defenses against biological weapons, which sometimes kind of involves doing similar things as you would be doing if you were actually developing biological weapons because you need to sort of think about what you're trying to defend against. I think some of that research probably should be done only in a classified setting and not published.”

0.62

Alignment is not fundamentally unsolved; rather the key question is how difficult it proves to be, and there is no necessary connection between cognitive capability and moral motivations—one can have very smart minds that are nice and care about humans, just as within human distribution some geniuses are nasty and others are prosocial.

factualhigh valuecontestednovelty 1/4durability 3/4· Nick Bostrom

“I know a lot of researchers also from OpenAI who think that alignment is fundamentally unsolved. I don't agree with that um and I actually haven't heard anybody arguing for that. I think the question is how difficult it is. Um in principle, there can be very smart minds that that are nice and care about humans. I mean, within the human distribution for what it's worth, we see you know, some human geniuses are nasty and evil and some are really nice and prosocial and friendly and there seems to be no necessary connection between some level of cognitive capability and and your motivations whether you care about other people.”

0.61

AI could enable a society to lock itself into permanent orthodoxy if advanced surveillance and AI systems allow governments to identify and suppress political dissent with such precision that no opposition becomes possible, creating a permanent lock-in on beliefs and values.

forecasthigh valuecontestednovelty 2/4durability 3/4· Nick Bostrom

“You've warned that AI could let a society lock itself into permanent orthodoxy, right?...Already now, you could easily, if you sort of are able to eavesdrop on everybody's conversations and read everybody's messages, you have this mass surveillance, right? If you have AI's kind of mining through that, I think you could probably build a pretty accurate view of what each person's uh political opinions are or their views of of the government and so forth. Um so, with sufficient investment, you could imagine if if there were somebody already sitting as as like if if there were a world dictator and they had this technology and invested heavily, then maybe they would be able to leverage that, too, uh entrench themselves.”

0.60

True information can be harmful in ways distinct from false information, creating 'information hazards' where publishing accurate knowledge about existential risks can enable actualization rather than prevention.

definitionhigh valuespeaker onlynovelty 3/4durability 4/4· Nick Bostrom

“I had this paper uh introducing the concept of an information hazard. Um which is like so there are a lot of obvious ways in which false information and lies can be harmful, right? Like, you know, somebody spews out false financial information, investors invest, and they all lose their money, and it's like um but more interesting from a theoretical point of view are cases where true information uh can be harmful”

0.60

Sentience with capacity for conscious experience and suffering is a sufficient condition for moral status, and there might also be alternative bases like having self-conception across time, life goals to achieve, and ability to form reciprocal relationships.

definitionhigh valuecontestednovelty 1/4durability 4/4· Nick Bostrom

“I think is if they are sentient and have conscious experiences. Um and the ability to suffer, I think that's a sufficient condition for having moral status. Um and most people would agree with that. I think there might also be an alternative basis for having moral status, which is maybe if you have a conception of self as existing through time. Um life goals that you're hoping to achieve. Uh maybe the ability to form reciprocal relationships with with other human beings.”

0.56

If simulators at a higher level of reality exist, they would be subject to physical constraints operating at their level of reality, including limits on computational resources, memory capacity, repair capability, and defense against external threats, making them fundamentally more constrained than an omnipotent God.

causalhigh valuespeaker onlynovelty 2/4durability 4/4· Nick Bostrom

“Simulators would presumably be subject to a lot more constraints than the laws of logic. Um, so, maybe God can do anything that is logically consistent, but simulators would be subject to the physical constraints operating at their level of reality. Um, they would have to build this computer. They would have to ensure that it has sufficient memory capacity. Uh, you know, they would need to repair it if it breaks down. There would be a limit to the number of these computations they could perform.”

0.55

Superintelligent AI systems could potentially be designed with transparency and engineered motivations, unlike human leaders whose stated campaign promises don't guarantee subsequent behavior, enabling creation of a 'perfect ruler' without self-interest, incapable of lying, and perpetually committed to constitutional values.

forecasthigh valuespeaker onlynovelty 2/4durability 3/4· Nick Bostrom

“There might also be increasing um ways to arrange things in in world that are not really available today. Um, because AI systems, if you imagine maybe some super intelligence being empowered to kind of run the show, it could be transparent and its motives could be engineered in a way that's not the case with any human. So, with a human, you maybe appoint somebody to be, you know, um, the the leader of a country and he's sort of based on what they're saying on the campaign trail, but that's not really a guarantee to what they will subsequently do once they're in power. Um, with an AI, in principle, once we solve alignment, you might actually be able to create sort of a perfect ruler that didn't have any self-interest and that could not lie and that was always going to remain, um, committed to upholding the constitution”

0.54

Continual learning would be a significant capability unlock for AI systems if it can be achieved, potentially enabling models to learn and improve from new data continuously rather than only from fixed training datasets.

forecasthigh valueestablishednovelty 0/4durability 2/4· Nick Bostrom

“continual learning would be a significant unlock if that happens. Um, perhaps later this year or next year.”

0.52

The vulnerable world hypothesis would require civilization stabilization through either extremely intrusive surveillance monitoring what everyone is doing in real time to prevent individuals from creating weapons, or strong global governance capable of solving coordination problems between nations.

causalhigh valuespeaker onlynovelty 2/4durability 3/4· Nick Bostrom

“if there were some easy way to make biological weapons that didn't require sophisticated lab equipment and so forth. Then in in if if we are unlucky in the world were like that, then it might be that the only way for a civilization uh not to be devastated would be if you had this extremely intrusive um surveillance mesh that continuously monitored what everybody was doing... on the other hand, you have for a different class of ways the world could be vulnerable, you might need something like um a global governance that would be strong enough to solve some of the global coordination problems”

0.52

The vulnerable world hypothesis proposes that the space of possible technologies is like drawing balls from a giant urn, and throughout history most technologies discovered have been beneficial or mixed, but there could be 'black ball' technologies whose default outcome would be to destroy the civilization that discovers them.

definitionhigh valuespeaker onlynovelty 2/4durability 3/4· Nick Bostrom

“the urn analogy is a sort of metaphor for our technological uh progress where we invent new technologies. There are sort of a set of possible technologies permitted by our physics. And we are pulling out these one after another from this giant urn of possible inventions. And um throughout history, for the most part, all these technologies we have discovered have been um beneficial. Um Some have been a mixed bag. Uh we haven't yet pulled out one, like the black ball in this analogy, that is such that it sort of by default destroys the civilization that discovers it.”

0.52

A world where superintelligent AI solves all instrumental problems would liberate humans from the structures imposed by necessity—work for income, maintaining health through effort, acquiring skills for future advantage—and would require rethinking what makes life meaningful and worth living.

forecasthigh valuespeaker onlynovelty 2/4durability 3/4· Nick Bostrom

“right now so much much of our problems is oriented of our lives um are structured around trying to solve various problems. So uh you need to get a paycheck, right? To pay the bill. Um then you need to take out the trash because otherwise your house starts to smell and then you need to do that and you need to do that. You need to brush your teeth so that your teeth don't don't rot. Um and then you need to go to school so you can later get a job. Like the there's like this kind of set of instrumental necessities really give a kind of structure to our lives. Um and a lot of that would go away at technological maturity. That would be easier. There would be shortcuts to all of these outcomes.”

0.52

Getting the ethics of digital minds right is a crucial challenge in the AI era that requires work from philosophers, AI researchers, and will eventually become a political issue requiring detailed thinking about what AI moral status means in practice.

normativehigh valuespeaker onlynovelty 2/4durability 3/4· Nick Bostrom

“getting the ethics of digital minds right. Um And it's one thing to say affirm the principle that yes, that could be AIs, maybe they will have some forms of moral status, and we should be nice to them. But then, exactly what does that mean in practice? Like, what precisely does this moral status amount to? What what do these different AIs want? What practically should we be doing differently if we want to be nice to AIs? That's like a big open uh field um where there needs to be work from, you know, philosophers, AI researchers, and and ultimately it will become a like a political issue as well. Um where just there's a lot of stuff that still needs to be thought through.”

0.52

The fact that we are not seeing glitches in reality is weak evidence against the simulation hypothesis because superintelligences running simulations would have the capability to prevent detectable glitches from being observed by creatures inside the simulation, or even edit out memories of glitches afterward.

causalhigh valuespeaker onlynovelty 2/4durability 3/4· Nick Bostrom

“the fact that we are not seeing glitches is some evidence against the simulation hypothesis. It's just very weak evidence because it would be likely that, uh, these superintelligences that would be responsible for the vast majority of simulations that would ultimately be run would also have the capability of preventing glitches of a sort that would be detectable by the creatures, the observers inside the simulations. Uh, so, if they if they didn't want us to notice any glitches, I think they would be able to prevent us from doing so or or they could even edit out our memories afterwards if necessary.”

0.52

If a technology democratizes destructive power by empowering individuals to create unacceptable harm, cities could not exist in a world where everybody has the power to destroy cities, because in any sufficiently large population of millions, someone will be evil, mad, or attempting extortion.

causalhigh valuespeaker onlynovelty 1/4durability 4/4· Nick Bostrom

“it would certainly have been a bad scenario if because it would have democratized the power of mass destruction. So probably you couldn't have cities if everybody had the ability to destroy a city because in any sufficiently large population of several million people, there will always be somebody, right, who is either evil or goes mad or is trying some extortion thing.”

0.49

Why we find ourselves alive right now in 2026, at the threshold of superintelligence development, seems peculiar given that most humans lived long ago as hunter-gatherers or farmers, and this timing question opens up interesting possibilities about simulation motives.

normativehigh valuespeaker onlynovelty 2/4durability 2/4· Nick Bostrom

“Why do we find ourselves alive right now in 2026? This seems to be a kind of peculiar time given that we're sitting just at the threshold to the development of superintelligence. Um most people uh lived a long time ago. Uh maybe they were hunter-gatherers or farmers in Egypt or something like that, right? And and right now we are sitting here talking about the prospect of this like most important thing in every human history, and we are really close to it.”

0.48

The question with highest information value for decision-making is whether it would be wise to slow or accelerate AI development, or whether to increase or decrease government involvement in AI governance.

factualhigh valuespeaker onlynovelty 1/4durability 3/4· Nick Bostrom

“would it be wise for me to try to like slow or to accelerate AI development? Like would it be wise to try to get governments more involved or less involved? Would it be like maybe some questions of that sort? Um would be have it have the highest information value.”

0.48

A simulator creating the universe is different from philosophical perspectives treating physics as information processing, because simulation requires a literal physical computer at the simulators' level of reality running code to implement our world, whereas informational physics is just a theoretical framework choice.

definitionhigh valuespeaker onlynovelty 1/4durability 3/4· Nick Bostrom

“It's not meant to be a metaphor, but a in a literal sense if the third hypothesis is true. Um then and and the world around you, the uh the microphone that that you're looking at there, you know, your sweater, all of this stuff um exists as information being processed inside the computer that is built by some advanced civilization somewhere. And it has some power supply and it's, you know, generates heat... it's not different from the idea which with which it is sometimes confused that like, you know, there's like a kind of perspective where you could look at fundamental physics as in information processing terms.”

0.48

Earlier work from the 1990s on superintelligence showed only a low-resolution view that superintelligence would be important and probably take neural network form, but current understanding has much finer granularity showing exactly how AI is unfolding, revealing an extended period of roughly human-level AI systems before superintelligence.

factualhigh valuespeaker onlynovelty 1/4durability 3/4· Nick Bostrom

“It the changes are more in terms of um getting a higher resolution view... like a very low-res version of just kind of some pixelated um map... as more bits come down, it kind of gets all photorealistic. And similarly back in the '90s when I uh you know, some of my early work uh like that that was like a kind of very um low-res version... so a lot of that, but now of course we can see much finer granularity, like exactly how things are unfolding.”

0.48

Some simulation motives beyond entertainment are possible—simulators might be doing ancestor simulations for historical research, scientific study, or other purposes that are difficult to predict, making motive-based inference about how to act in a simulation largely speculative.

forecasthigh valuespeaker onlynovelty 1/4durability 3/4· Nick Bostrom

“It's difficult to know exactly what would make most sense if you sort of conditionalized your actions on the simulation hypothesis. I would say that to a first rough initial approximation, probably you should just keep going on as you would have anyway... We are fairly ignorant about the sort of psychology and the motivations of the simulators, although one can uh, entertain various speculations. I I would be hesitant to um, draw any strong inferences from that.”

0.48

The simulation argument is distinct from other epistemological skeptical scenarios like Descartes' demon or the dream problem because it starts from empirical assumption that physics and technology are real and reasons through technological progress rather than questioning whether external reality exists at all.

definitionhigh valuespeaker onlynovelty 1/4durability 3/4· Nick Bostrom

“the simulation argument starts from the assumption um, that everything is as it appears to be... But the simulation argument starts from the assumption um, that everything is as it appears to be. There is this physical world... And then, thinks through what the consequences of that is... skeptical speculations are old... ancient Chinese philosophy and like Descartes... prove to me that the external world exists... the simulation argument starts from the assumption um, that everything is as it appears to be”

0.48

Humans should not necessarily try to be interesting to simulators in a simulation, since it is unclear that entertainment is the most likely motive, and optimally one should continue living much as one would in baseline reality since the same basic needs and actions are required either way.

normativehigh valuespeaker onlynovelty 1/4durability 3/4· Nick Bostrom

“I wouldn't necessarily say that you should. Um, it's not clear that entertainment is the most likely motive um, if we're running a simulation. It's it's difficult to know exactly what would make most sense if you sort of conditionalized your actions on the simulation hypothesis. I would say that to a first rough initial approximation, probably you should just keep going on as you would have anyway. Like, because even inside the simulation you still need to perform various actions in order to sort of sustain yourself. Um, for the same reasons as you would in a basement reality.”

0.48

A fundamental creator God and a simulator creating the universe are logically consistent possibilities—God could have created the advanced civilization that then builds the simulator, just as biological humans breed ants in jars despite being themselves created.

definitionhigh valuespeaker onlynovelty 1/4durability 3/4· Nick Bostrom

“Now, that's uh it's consistent with there being a kind of more fundamental creator. Like so, like God might have created everything, including this advanced civilization that is then building the computer on which we run. Like just as, you know, you can you know, breed a bunch of ants in in a glass jar. Like in some sense, maybe you created us ants by breeding them. But still, you know, your parents created you. And and maybe uh the sort of divine creator created the whole thing. So, it's it's like I've logically consistent with that either being the case or not being the case.”

0.48

First-strike advantage creates a vulnerability where powerful actors face incentives to create great harms—a different class of black ball from democratized destructive power, where governance structure rather than capability restriction becomes the solution.

causalhigh valuespeaker onlynovelty 1/4durability 3/4· Nick Bostrom

“there could also be more subtle ways in which the world could be vulnerable. Um not by empowering individuals to create great harms, but by um technologies that create incentives for powerful actors uh to create great harms. Like if there is a strong first strike advantage, for example.”

0.43

Even the wealthiest people cannot currently purchase freedom from aging, sickness, or fitness maintenance through money, meaning post-superintelligence utopia would make available things currently impossible to acquire at any price, making it more attractive than inheriting wealth.

factualhigh valuespeaker onlynovelty 1/4durability 3/4· Nick Bostrom

“there are a lot of things in today's world that money can't buy. Um for example, um health only to a very limited degree can you improve your health with by spending money on it? Like the richest people also get old and sick and you know, if they want to be fit, they still have to go to the gym. That's one example. Um many other things are currently just not purchasable at any price and that but they they would become available in in this scenario.”

0.27

Even if language models plateau with permanent jaggedness (capability curves with dips), the parts where the jaggedness moves upward will eventually still exceed the full capability set of human minds.

forecastcontestednovelty 1/4durability 2/4· Nick Bostrom

“I think eventually um even if there remains jaggedness, the parts where the jaggedness goes inward will still s- exceed uh the disk of what what the human mind is capable of.”

0.26

If we are living in a simulation, an afterlife becomes a much more likely prospect because simulation termination need not result in permanent cessation—the simulators could preserve, backup, or continue our consciousness in other contexts.

causalspeaker onlynovelty 1/4durability 2/4· Nick Bostrom

“if we're in a simulation, I think, for example, an afterlife is uh a much likelier prospect... after if if one simulation terminates, we need to make more progress on AI alignment”

0.21

Historically, scenarios with extremely fast AI takeoff and scenarios with extremely slow takeoff each had some probability, but there was never an overwhelming probability on either, with probabilities distributed across multiple timeframes from rapid to very slow development.

factualspeaker onlynovelty 1/4durability 2/4· Nick Bostrom

“I I that that there were scenarios where the takeoff would go slowly and and once were it might happen more quickly. Somebody is sitting tinkering with their, you know, computer figuring out the key algorithm and things kind of going through more of a short period of time. But And And there was some probability on on this this this kind of scenario as well, but not an overwhelming probability relative to it either going much much slower or much faster, more likely faster.”