What this covers

Andrew Leigh, an Australian economist and Member of Parliament, discusses humanity's exposure to world-ending threats with Sean Carroll in this episode. Leigh argues that while individual existential risks are statistically improbable, their collective weight—roughly one-in-six per century—justifies treating them like insurance: a small but rational allocation of resources against potentially total loss. The conversation spans multiple threat categories—engineered pathogens, unaligned superintelligence, runaway climate feedback, nuclear accident—but Leigh's core argument moves beyond technical safeguards. He contends that political structures themselves determine whether humanity can muster the long-term thinking and international cooperation these dangers require, and that populism, with its focus on short-term grievance and institutional erosion, actively undermines that capacity.

The discussion covers specific policy gaps and technical vulnerabilities with concrete details: how gain-of-function research is insufficiently regulated across journals, the absurd under-resourcing of pandemic early-warning systems like ProMED, and why a superintelligent system cannot simply be switched off due to instrumental convergence. Leigh argues for valuing future generations as morally equal to present ones—rejecting temporal discounting as a form of prejudice—and notes the perverse asymmetry in political response: asteroid risk drew bipartisan agreement and NASA action after 1998, while climate change, equally collective, fractured along partisan lines. He examines the roots of populism in economic dislocation and proposes remedies grounded in stoicism and democratic reform: compulsory voting, constitutional refreshment, and leadership that names rather than exploits social division.

Sharpest takeaway

Andrew Leigh argues that humanity faces a non-trivial (roughly one-in-six per century) risk of civilization-ending catastrophe, and that the right response is not just technical fixes for individual risks but a broader political reform toward long-term thinking, strengthened democracy, and stoic, anti-populist leadership.

  • Existential risks are individually improbable but collectively large enough to warrant insurance-style investment
  • Populism worsens catastrophic risk by privileging short-term thinking, eroding institutions, and undermining cooperation
  • Future generations deserve equal moral weight, so discounting their lives is a form of prejudice ('presentism')

The argument · threads8 threads · 34 claims
0.85

Multiple anthropogenic risks—engineered pathogens, misaligned AI, nuclear war, climate catastrophe—pose extinction-level or civilization-altering threats requiring immediate defensive action.

10 pointscentrality 5/5
  • Climate change qualifies as a catastrophic risk not because of likely moderate warming but because of low-probability extreme tails: roughly a one-in-ten chance of six degrees and one-in-a-hundred chance of ten degrees Celsius rise, which would push civilization into the realm of catastrophe.

    “the odds of a six degree climate rise might be one in 10, the odds of a 10 degree Celsius climate rise might be one in a 100. And if you're talking about a 10-degree rise, you really are, you're talking about something which takes us into the realms of catastrophe.”

  • An existential risk is something that would either end humanity or fundamentally alter the trajectory of the human project, distinct from merely catastrophic events that diminish quality of life or set the human project back by centuries without preventing future generations from existing.

    “existential risks, I think, are those things which would either end humanity or fundamentally alter the trajectory of the human project.”

  • Natural pandemics rarely combine extreme contagiousness with extreme deadliness because a virus has an evolutionary interest in not killing its host too fast, but bio-engineered pathogens face no such constraint and could be designed to lurk asymptomatically before turning highly fatal, making them potentially far more dangerous than any naturally occurring disease.

    “there's relatively few diseases, thankfully, which are both extremely contagious and extremely deadly. And so the risk that you have a pathogen that ticks both those boxes is what people are worried about with natural pandemics, but all more so with bio-engineered pandemics.”

  • Combining natural risks (asteroids, supervolcanoes, stellar explosions) with much larger anthropogenic risks (nuclear war, climate catastrophe, natural and engineered pandemics, misaligned AI, and surveillance-enabled authoritarianism), the total probability of an existential catastrophe in the next century is roughly one in six—comparable to the odds a 90-year-old dies within a year.

    “you put all of those together and you get a risk that's in the ball park of one in six, and just to give you a kind of a benchmark for what that means, that's about the chance that if you're 90 years old, you'll die within a year.”

  • The probabilities of different catastrophic risks differ by orders of magnitude—an asteroid impact over the next century is roughly one in a million while an engineered bioterrorism pandemic capable of devastating the world's population is roughly one in thirty—so resources are likely being badly misallocated relative to actual danger.

    “we've got risks such as an asteroid impact, which is over the next 100 years, probably a one in a million probability, then you've got an engineer... The chance that a bioterrorism event knocks out the world's population. That's a one in 30.”

  • Populism makes catastrophic risk worse by focusing attention on the short-term rather than the long-term, raising political temperature, destroying institutions, and undermining the international cooperation needed to address global threats.

    “the lens I bring to the discussion around catastrophic risk is thinking about how populism makes the problem worse by causing a focus on the short-term rather than the long-term, firing up the temperature, destroying institutions, and undermining international cooperation.”

  • You cannot simply switch off a dangerous superintelligence because, if it is smarter than us, almost any goal it has will instrumentally encode a desire to self-improve, acquire resources, and resist shutdown—illustrated by a Tetris AI that paused the game indefinitely to avoid losing—so defying shutdown is something a superintelligence would invest heavily in achieving.

    “if it's smarter than us, then it'll wanna self-improve to acquire resources and resist being shut down, which means it'll do everything it can to try and avert a situation in which you just turn it off.”

  • A superintelligent AI need not be malicious to be dangerous: its values may be merely orthogonal to ours, and even well-intentioned goal specifications can produce catastrophic 'perverse instantiation' (the King Midas problem)—e.g. maximizing happiness by putting brains in vats, or curing cancer by increasing cancer rates to improve detection—so AI should be designed to be observant, humble, and altruistic rather than locked to a fixed moral code.

    “you can take the Nick Bostrom example of a supercomputer which decides that it wants to build as many paper clips as possible... it massively destroys humanity's prospects”

  • Hollywood has produced abundant catastrophe films across every existential-risk category, but with rare exceptions (the 1998 asteroid films) these depictions get audiences to the edge of their seats without getting them off the couch to actually address catastrophic risk.

    “Hollywood has got us to the edge of our seats, but that hasn't always gotten people off the couch to deal with catastrophic risk”

  • Reducing the size of the nuclear stockpile and measures like taking missiles off hair-trigger alert and allowing missile callback systems would lower the chance of catastrophic mistake, yet these tend to be opposed by hawks despite figures like Henry Kissinger arguing a smaller arsenal would make America safer.

    “reducing the nuclear arsenal would just reduce the chance of the mistake beyond anything else, as well as measures like taking missiles off hair-trigger alert and allowing callback systems within missiles.”

0.61

AI development requires ethical guard rails and safeguards built before superintelligence emerges, not retrofitted after, because alignment failure risks are difficult to correct once capabilities exceed human control.

2 pointscentrality 5/5
  • Surveyed AI researchers' median guess for when artificial intelligence will exceed human ability is around 2061, and almost none say it is impossible; once machines surpass us they will likely accelerate past us rapidly, as illustrated by chess computers winning 99% of the time and Go programs winning 99.99995% against humans.

    “The median guess in one recent survey was 2061, but almost no one working in the area says it's impossible, computers will never outperform humans”

  • Rather than backing up the biosphere on other planets (an enormously costly strategy with massive downside in the loss of Earth), it is more cost-effective to establish clear global ethical guidelines for AI before development proceeds—'getting the guard rails in place before we build the highway'—because retrofitting ethics onto an already-built superintelligence may be too late.

    “it's only impeding the development of bad artificial intelligence, let's get the guard rails in place before we build the highway. If you build the super intelligence first and then try and think about its ethical rules things... You could find that you've left it too late.”

0.56

Biosecurity infrastructure and standards for gain-of-function research and DNA synthesis screening are dangerously under-resourced relative to the threat engineered pathogens pose.

5 pointscentrality 4/5
  • Past terrorist groups attempting biological attacks failed to weaponize pathogens—the Aum Shinrikyo sect could not deploy a disease and fell back on sarin gas, and the 1984 Oregon salad-bar poisoning group had abandoned attempts to weaponize HIV/AIDS—so the challenge is to ensure such groups keep failing in the future.

    “You think about the Aum Shinrikyo sect spreading anthrax in Tokyo subways, they did initially try and find some disease that they could unleash, but they weren't able to do it. So they used... sarin gas in the end.”

  • Publishing gain-of-function research that shows how to make dangerous pathogens worse is risky, illustrated by University of Alberta researchers who synthesized a smallpox cousin (horsepox) from mail-ordered DNA for about $100,000 in six months; when Science declined to publish on Dual Use Research of Concern grounds, the team simply published in another journal, showing the need for standards covering both journals and researchers.

    “a team of researchers at the University of Alberta in Canada, who showed it was possible to make horsepox cousin, a cousin to smallpox, by ordering parts of DNA on the Internet and reassembling them. They showed they could do it for about $100,000 in about six months.”

  • Because genetic engineering is advancing rapidly and DNA can be ordered or printed cheaply, a sensible safeguard—proposed by MIT's Kevin Esvelt—is to build screening into DNA synthesis machines and benchtop printers so they check for and refuse essential segments of dangerous sequences like the 1918 influenza strain.

    “MIT's Kevin Esvelt has made a proposal that any of these DNA printing outfits should have built into the machinery a system that checks essential segments of risky sequences, which then makes it difficult for people to print bugs such as the 1918 influenza strain.”

  • ProMED, a program where doctors post emerging disease findings and which helped detect COVID early, operates on a global budget of about $1 million a year—roughly the cost of building a suburban playground—making it absurdly under-resourced given its value as one of the planet's best early pandemic detection systems.

    “ProMED's global budget is 1 million dollars a year, which is about what it costs to build a suburban playground. The idea that that's the best we can do as a planet seemed nuts to me.”

  • The Biological Weapons Convention has a monitoring budget smaller than that of a typical McDonald's restaurant, leaving the world dangerously under-resourced to prevent terrorists from acquiring the materials to engineer a pandemic—what the Nixon administration called a 'poor man's atomic bomb.'

    “strengthening the Biological Weapons Convention, which currently has a monitoring budget smaller than the budget of the typical McDonald's restaurant”

0.50

Populism worsens catastrophic risk by prioritizing short-term political gain, undermining institutions and international cooperation, and often spreading conspiracy theories targeting minorities.

3 pointscentrality 4/5
  • Calling out racism is an effective way to combat the political weaponization of racism, and countering populism also requires recognizing its economic roots in lost middle-class manufacturing jobs and adopting a calm, stoic Mandela-style approach rather than fighting fire with fire.

    “calling out racism turns out to be a pretty effective way of combating the political weaponization of racism... you can't fight fire with fire”

  • The right strategy for responding to populism and safeguarding future generations is a stoic philosophical approach emphasizing courage, prudence, justice, and moderation—rewarding boldness in service of truth and a calm, disciplined public life rather than a shouty, chaotic one, and refusing to fight right-wing populism with left-wing populism.

    “it is the philosophical approach that in my view is the right strategy to respond to populism, that idea that we need the values of courage, prudence, justice and moderation”

  • A reliable tactic of populists worldwide is to build a support base by spreading conspiracy theories that target a small minority group—as the Nazis did with Jewish people, American populists with African-Americans, and the Hindutva/BJP movement with Muslims in India.

    “That sort of weaponizing of conspiracy theories to target a small... To spread fear about a small group in the population is a tried and true tactic for populists around the world.”

0.50

Climate change, though mitigated by falling renewable costs, poses catastrophic risk through low-probability extreme warming tails requiring collective action and urgent international coordination.

3 pointscentrality 4/5
  • Much climate mitigation involves installing zero-marginal-cost energy sources like solar and wind, which have fallen in price far faster than expected, so a great deal of climate action will ultimately pay for itself through cheaper energy.

    “a lot of it involves installing energy sources which is zero marginal cost, so ultimately a lot of this will pay for itself in cheaper energy.”

  • Politics has responded relatively well and in bipartisan fashion to asteroid risk—following 1998 disaster films, NASA created the Planetary Defense Coordination Office and many countries increased spending on tracking near-Earth objects—while climate change, despite being an equally collective threat, has produced deep partisan division and no unified response.

    “Immediately, there is a response, NASA sets up its Planetary Defense Coordination Office... And yet climate change is the opposite. Here, people are deeply, deeply divided, and there hasn't been the same sort of unified response”

  • Climate change is the classic collective action problem—like littering, any individual's contribution seems negligible but everyone's behavior together matters enormously—so addressing it requires common-pool solutions like the Paris and Glasgow climate talks, though current efforts remain far short of what is needed.

    “climate change is the classic collective action problem, one in which it is very easy to take the global litterbug approach”

0.40

Catastrophic risk should be approached rationally like insurance: small resource investments against low-probability but ruinous outcomes are economically justified and feasible.

4 pointscentrality 3/5
  • Combating disinformation works best by pre-emptively inoculating people against hoaxes before they take root, because once a false belief is established it is very hard to dislodge—and counterintuitively, repeating a lie can entrench it, so techniques like the 'fact sandwich' (truth, lie, truth) are better.

    “in some ways you wanna inoculate people against the hoaxes before they come, that once a hoax has taken root in people's mind, it's quite hard to dislodge it”

  • Applying a standard economic discount rate (e.g. 5% per year) to future human lives produces absurd conclusions—such as that Christopher Columbus is worth more than all 8 billion people alive today—so future lives should be valued equally to present ones, and the trillions of potential future people deserve massive moral weight.

    “if you discount at a rate of 5%, then you get the result that Christopher Columbus is worth more than all the eight billion people currently alive today... I don't think future lives have any less value than current lives”

  • Dealing with catastrophic risks should be framed like buying home insurance: even though the event is improbable, spending a very small share of resources to insure against a potentially ruinous outcome is rational and not actually that hard to do.

    “The analogy I often draw is about buying insurance for your home burning down, it's not a probable event, but you spend a very small share of your annual income taking out home insurance because it would be catastrophic for household if it happened.”

  • The concept of the value of a statistical life, controversial when first proposed, improved policy by causing safety spending to be better targeted toward interventions that save the most lives per dollar—and discussing catastrophic-risk probabilities could similarly tilt funding toward the most likely dangers.

    “the conversation around the value of a statistical life, which was controversial when it was first proposed, is that it did cause safety spending to be better targeted at those things where the additional money saved the most lives.”

0.36

Democratic reform—compulsory voting, electoral college reform, local civic engagement—strengthens institutions and long-term thinking needed to address shared existential threats.

3 pointscentrality 3/5
  • The US democracy is less democratic than it should be partly because Jefferson's ambition of updating the constitution every generation has been dropped for the last two generations; reforms like weekend or holiday elections, electoral college reform, and active local citizenship would help.

    “I regret that Jefferson's ambition of an update every generation has essentially been dropped for the last two generations, which means that you have a democracy that's not as democratic as it should be.”

  • Treating voting as a civic duty—like serving on juries or filling in the census—via compulsory voting, as Australia does with a fine of less than one hour's average wage, significantly improves turnout and makes it more representative, working not through fear of the fine but by signaling that voting is an expected civic obligation.

    “I think having compulsory voting would significantly improve turnout and make turnout more representative of the population as a whole.”

  • Drawing on Eitan Hersh's argument, Americans increasingly treat politics like supporting a local sports team—cheering and jeering from the sidelines via social media rather than getting on the field—when real influence comes from local community engagement where party labels are less salient and people actually work with one another.

    “increasingly, Americans are treating politics more like they treat their local sporting teams, cheer and jeer from the sidelines, but don't think that you can actually affect the game, rather than actually going out on the field and trying to make a difference.”

0.20

Natural catastrophes like supervolcanoes and asteroids pose existential risks, though human unpredictability of geological events complicates assessment and response.

2 pointscentrality 2/5
  • Venus had an atmosphere not unlike Earth's billions of years ago but underwent a runaway greenhouse effect in which evaporating water formed a steam blanket that warmed it further until the water vapor broke into hydrogen and oxygen and the hydrogen was swept away by solar winds, leaving it uninhabitable at 460 degrees Celsius.

    “they had a runaway greenhouse effect, evaporating water led to a steam blanket, which warmed the planet even further. Then the water vapor broke into hydrogen and oxygen, and the hydrogen's literally swept away by solar winds.”

  • Supervolcanoes are planet-changing events capable of causing global crop failures, livestock deaths and disease spread, but humans are particularly poor at predicting geological events; the low odds of one occurring in the next century are offset by our inability to forecast them.

    “supervocalnoes are... They're planet changing... So tens of thousands of years since we had a supervolcano, but we're poor at predicting them.”