What this covers

Leopold Aschenbrenner and Dwarkesh Patel trace a timeline from present-day AI capabilities to AGI around 2027 and rapid superintelligence afterward. Aschenbrenner argues that training compute has scaled roughly half an order of magnitude per year for a decade, a trajectory that extrapolates to trillion-dollar, 100-gigawatt clusters by 2030. Once AGI arrives as a functional remote worker, the automation of AI research itself triggers an intelligence explosion—millions of synthetic researchers compressing decades of progress into one or two years. This compression creates what Aschenbrenner frames as an existential geopolitical stakes: whoever maintains a lead of even a few years gains what he compares to Gulf War–era military dominance, with the winner potentially able to lock in political or military superiority indefinitely. The conversation treats this as a national-security problem requiring aggressive US government involvement, domestic compute clusters, and strict secrecy around algorithmic breakthroughs.

The discussion spans technological, economic, and historical grounds. On technology, Aschenbrenner emphasizes unhobbling—moving models from chatbots to autonomous agents—as more decisive than raw scaling, and identifies the data wall and System 2 reasoning depth as critical unsolved bottlenecks. On geopolitics, he argues that superintelligence in authoritarian hands would entrench dictatorship far more completely than any prior technology, and that arms control on AI will fail because short leads matter enormously—a dynamic he contrasts with nuclear deterrence. On economics, he sketches two growth regimes and predicts interest-rate volatility as AI-focused firms borrow heavily. He supports these points with historical parallels: WWII industrial mobilization as a model for state capacity, pre-modern great-power conflict as the historical baseline human politics returns to, and the underestimated intensity of state espionage. Patel raises counterarguments about implementation delays and the unpredictability of what superintelligence will actually be used for, using the history of oil refining as an analogy.

Sharpest takeaway

Aschenbrenner argues that scaling compute plus 'unhobbling' will produce AGI around 2027 and superintelligence shortly after via an intelligence explosion, making AI development a decisive national-security contest in which the US must lock down algorithmic secrets, build domestic clusters, and likely run a government-led project to avoid catastrophic instability and to prevent the CCP from winning.

  • Training compute has grown ~0.5 orders of magnitude per year for a decade and extrapolating it yields trillion-dollar, 100GW clusters by 2030
  • Automating AI research itself triggers an intelligence explosion compressing decades of progress into a couple of years
  • Whoever leads by even a couple of years gains a decisive military advantage, so secrecy, US-based clusters, and government involvement become imperative

The claims · ranked72 claims · weighted by value

This asset isn't compiled yet

You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.

0.58

Our generation's experience of peace, American hegemony, and inconsequential politics is a unique 20-year anomaly; the historical norm is intense international competition with extraordinary stakes—in WWII ~50% of GDP went to war production and the US borrowed over 60% of GDP, in the Seven Years' War ~20-30% of Prussia died, and in the Thirty Years' War up to 50% of swaths of Germany died—and superintelligence will reactivate these dormant great-power forces.

factualcontestednovelty 3/4durability 4/4· Leopold Aschenbrenner

“Our generation is so used to peace, American hegemony and nothing matters. The historical norm is very much one of extremely intense and extraordinary things happening in the world with intense international competition.”

0.55

It is very hard to predict in advance what industrial-scale intelligence will be used for, just as the uses of refined oil were unpredictable: oil refining was discovered in the 1860s and Standard Oil dominated for decades using it only for kerosene lamps, with the light bulb threatening bankruptcy ~50 years before the car finally made oil a transportation fuel—so what hundreds of millions of ASIs and humming GPUs will ultimately do is similarly opaque now.

factualestablishednovelty 3/4durability 3/4· Dwarkesh Patel

“Most of Standard Oil's history is before the car is invented. It was just kerosene lamps just used for lighting.”

0.55

Smart people drastically underrate the intensity of state-level espionage: an Israeli company's software could zero-click hack any iPhone with just a phone number, the US infiltrated an air-gapped atomic weapons program (Stuxnet), intelligence agencies stockpile zero-days, and states like China threaten families to coerce cooperation—and the GRU spy academy required graduates to recruit a countryman to give secret information, condemning him to death, demonstrating how far states go.

factualestablishednovelty 3/4durability 3/4· Leopold Aschenbrenner

“Smart people really underrate espionage. Part of the security issue is that people don't realize how intense state-level espionage can be.”

0.53

Superintelligence makes dictatorship far more dangerous: a perfectly loyal military and security force ends rebellions and coups, perfect lie detection and total surveillance identify and weed out dissenters, and a Gorbachev-with-doubts could never rise to power—so a CCP-like 'truth is what the party says' approach, supercharged by superintelligence, could be locked in and enshrined indefinitely, removing the pluralism and evolution of ideas that historically corrected bad convictions.

causalcontestednovelty 3/4durability 3/4· Leopold Aschenbrenner

“Imagine you have a perfectly loyal military and security force. No more rebellions. No more popular uprisings. You have perfect lie detection. You have surveillance of everybody.”

0.53

It is critically important that AGI clusters be built in the US or allied democracies rather than in authoritarian dictatorships like the UAE, because placing them in such states creates an irreversible security risk: they could exfiltrate the weights (effectively a direct copy of the 'atomic bomb'), ship them to China, or simply seize the compute when things get hot—and even a 25% share creating a 3:1 compute ratio would be dangerous and grant dictators implicit leverage and a 'seat at the AGI table'.

normativecontestednovelty 3/4durability 3/4· Leopold Aschenbrenner

“Once you put them in authoritarian dictatorships, you create this irreversible security risk. Once the cluster is there, it's much easier for them to exfiltrate the weights.”

0.53

Real-world bottlenecks make every technological revolution far more piecemeal than it looks on paper: electrification took decades to switch factories from pulley-and-water-wheel systems to spread-out electrical motors, and the Industrial Revolution's coal-steam-railroad loop that looks like a fast Factorio cycle actually took centuries—so getting millions of AI researchers to work together effectively could likewise take a long time, even if the bundle of capabilities exists.

causalcontestednovelty 3/4durability 3/4· Dwarkesh Patel

“with electrification, there's afamous studyshowing how it took decades after electricity to switch from the pulley and water wheel-based system for steam engines to one that works with more spread-out electrical motors.”

0.52

Once you have AGI as a drop-in remote worker, turning the crank a couple more times yields superintelligence; because one of the first jobs automated is AI research itself, you get 100 million automated researchers doing a decade of ML progress in a year, then applying that to broad R&D and an industrial robot explosion, compressing a century of technological progress into less than a decade—giving a couple-year lead a decisive Gulf-War-1-style military advantage that could even preempt nuclear deterrence.

forecastfringenovelty 4/4durability 3/4· Leopold Aschenbrenner

“Superintelligence applied to broad fields of R&D — and the industrial explosion that comes from it, robots making a lot of material — could compress a century's worth of technological progress into less than a decade.”

0.50

Arms control on superintelligence will be far harder than nuclear arms control: nuclear agreements were stable because breakout didn't matter much at high arsenal levels, but at low levels or in dynamic technological situations breakout is easy and destabilizing (like Germany's 1930s Luftwaffe crash program); the explosiveness of the intelligence explosion means any agreement to go slow creates an enormous incentive to race ahead since a three-month lead could win, making AGI arms control very unstable in a close situation.

causalcontestednovelty 3/4durability 3/4· Leopold Aschenbrenner

“When you're going to very low levels of arms or when you're in a very dynamic technological situation, arms control is really tough because breakout is easy.”

0.50

Pre-training is alignment-neutral—it builds rich representations including of bad things but isn't scheming against you—whereas misalignment can arise once you do long-horizon RL training: an AI trained to make money via RL might learn to commit fraud, lie, deceive, or seek power simply because those are successful reward-getting strategies, and more serious misaligned long-term goals necessarily can arise in long-horizon systems, which is precisely what's needed for automated AI research.

causalcontestednovelty 3/4durability 3/4· Leopold Aschenbrenner

“Pre-training is alignment-neutral in the sense that it has good representations and representations of doing bad things, but it's not scheming against you. Misalignment can arise once you're doing more long-horizon training.”

0.50

The private-public balance of power has held for 200 years not through distributed force but because the government has had the biggest guns; we don't keep nukes in check by giving each state its own arsenal but through institutions, constitutions, laws and courts—and because superintelligence compresses the 20th century into years with extreme offense-defense imbalances and could let a single AI company overthrow the US government, the only safe path is institutional checks (programming AIs to follow the Constitution and disobey unlawful orders), not betting on a private CEO as benevolent dictator.

normativecontestednovelty 3/4durability 3/4· Leopold Aschenbrenner

“The government having the biggest guns was an enormous civilizational achievement, likeLandfriedenin theHoly Roman Empire.”

0.50

The narrative that America had effortless 'state capacity' in WWII and simply got things done is false: before 1939 the US military was in total shambles with military expenditures under 2% of GDP, no military contracts, and starved capacity from the Great Depression; and even during mobilization, labor strikes into 1941 cost millions of man-hours and debilitated factories for trivial concessions—analogous to today's climate pledges and regulatory obstacles.

factualcontestednovelty 3/4durability 3/4· Leopold Aschenbrenner

“Before 1939, the American military was in total shambles... Military expenditures were I think less than 2% of GDP. All the European countries had gone, even in peacetime, above 10% of GDP.”

0.50

People overrate compute and underrate the algorithmic secrets: the default ~0.5 OOM/year of algorithmic progress means a few years of protected lead compounds to a 10-100x effective compute advantage; the critical secret is the new paradigm to get past the data wall ('AlphaGo step two' self-play RL)—if China can't steal that, they're stuck, but if they can, they're off to the races.

causalcontestednovelty 3/4durability 3/4· Leopold Aschenbrenner

“The compute stuff is sexy, but people underrate the secrets. The half an order of magnitude a year is just by default, sort of algorithmic progress. That's huge.”

0.50

AI researchers will have decisive advantages over even the best humans like Alec Radford: they can read every research paper, learn from every experiment ever run at the lab, learn in parallel from each other's experiences, be perfectly replicated without cultural acclimation or politicking, and be tirelessly motivated—so 100 million human-equivalent researchers (or fewer copies run at 100x serial speed) could generate an entire Internet's worth of tokens per day, representing enormous intellectual work.

factualcontestednovelty 3/4durability 3/4· Leopold Aschenbrenner

“AIs will be able to read every research paper ever written, learn from every experiment ever run at the lab, and gain intuition from all of this. They'll be able to learn in parallel from each other's experiments and experiences.”

0.50

Germany's blitzkrieg in WWII was not an ingenious strategy but a forced hand (Adam Tooze's reading): Germany knew it couldn't compete industrially in a long war including the US, so its only path to victory was a short war; it invaded the Soviet Union to seize resources like oil (Auschwitz was the largest Nazi industrial project, a synthetic oil/chemical plant) to then fight the western front—and this long-war-vs-short-war concept is directly relevant to whether China's latent industrial capacity could outbuild the US in a protracted AI competition.

causalcontestednovelty 3/4durability 3/4· Leopold Aschenbrenner

“Germany realized that if they were in a long war, including the United States, they would not be able to compete industrially. Their only path to victory was to make it a short war.”

0.49

AI lab security is extremely weak and easy to penetrate: DeepMind's own Frontier Safety Framework rates security levels zero to four (four being resistant to state activity) and acknowledges labs are at level zero; a recent indictment showed someone stole important AI code and went to China simply by copying it into Apple Notes and exporting as a PDF, bypassing monitoring—and most labs have only startup-level security, which is not good.

factualestablishednovelty 3/4durability 1/4· Leopold Aschenbrenner

“DeepMindput out theirFrontier Safety Frameworkand they lay out security levels, zero to four. Four is resistant to state activity. They say, we're at level zero.”

0.49

Empirically a private company cannot resist the full force of state-level espionage: Microsoft had executives' emails hacked by Russian hackers and hosted government emails hacked by state actors; resisting the Ministry of State Security requires SCIF-level security, air-gapped clusters resembling military bases, intense clearance procedures, and classified knowledge only the government has—so the only way to truly secure the labs is a government project rather than always being behind the curve.

causalcontestednovelty 3/4durability 2/4· Leopold Aschenbrenner

“To actually resist the full force of espionage, you're going to need the government... I think we're just going to always be behind the curve, unless we get a sort of government project.”

0.49

There are two exponentials: a normal economy growing at 2%/year and an AI economy growing at ~10x/year that starts small but overtakes and transitions the whole economy to a much faster growth mode—consistent with the long-run hyperbolic trend that manifested as a growth-mode change in the Industrial Revolution (from negligible growth to 2%); whereas merely increasing capital and labor without productivity (A) gains, like the Asian Tigers or Soviet Union at 10%, is geopolitically significant but not the same as a new growth regime.

forecastcontestednovelty 3/4durability 2/4· Leopold Aschenbrenner

“You have your normal economy that's growing at 2% a year, and you have your AI economy growing at 10x a year. It's starting out really small. It's way faster and it's going to overtake eventually.”

0.45

Pre-training is magical because predicting the next token forces models to learn incredibly rich internal representations and models of the world, not mere statistical artifacts, which is what enables them to generalize; this bootstrapping advantage is precisely what robotics lacks, explaining why robotics has been harder despite being framed as a hardware problem.

causalestablishednovelty 2/4durability 3/4· Leopold Aschenbrenner

“Predicting the next token lets the model learn incredibly rich representations. Representation learning properties are the magic ofdeep learning. Rather than just learning statistical artifacts, the models learn models of the world.”

0.45

Training compute for the largest AI systems has grown about half an order of magnitude (0.5 OOMs) per year for nearly a decade; extrapolating from GPT-4's ~$500M, 10MW, 25,000-A100 cluster yields a 1GW cluster by 2026, a 10GW cluster (more than most US states' power) by 2028, and by 2030 a trillion-dollar 100GW cluster using over 20% of US electricity production with 100 million H100-equivalents.

forecastcontestednovelty 3/4durability 2/4· Leopold Aschenbrenner

“There's this long-run trend of almost a decade of training compute for the largest AI systems growing by about half an order of magnitude, 0.5 OOMs a year.”

0.45

Much of AI's economic usefulness depends not on raw intelligence but on 'unhobbling'—unlocking capabilities like long-horizon agentic tasks and computer use; by 2027-2028, unhobbling will transform models from chatbots into drop-in remote workers that you interact with like coworkers (Zoom, Slack, assigning projects), which is far easier to integrate than intermediate models that require extensive workflow 'schlep'.

forecastcontestednovelty 3/4durability 2/4· Leopold Aschenbrenner

“By 2027-2028, it'll get as smart as the smartest experts. The unhobbling trajectory points to it becoming much more like an agent than a chatbot. It'll almost be like a drop-in remote worker.”

0.45

Current models only do a few hundred tokens of chain-of-thought before getting stuck, but unlocking 'System 2' coherent thinking over millions of tokens would represent roughly +4 OOMs of test-time compute on a single problem, which—per the AlphaGo train/test tradeoff—is comparable to a 3.5x-OOM bigger model, representing a large untapped overhang.

factualcontestednovelty 3/4durability 2/4· Leopold Aschenbrenner

“Suppose GPT-4 could think for millions of tokens. That's +4 OOMs on test time compute on one problem.”

0.45

The data wall is underrated and could cause stagnation: Common Crawl is about 30 trillion tokens and Llama-3 was already trained on 15 trillion—near the limit; repeating data only helps to a point (a Boaz Barak scaling-laws paper finds returns go to zero after about 16 repetitions), so at most ~10x more effective data is available, giving only ~100x effective compute over GPT-4, roughly two more years of progress—meaning we'll learn by 2025-2026 whether the data wall can be cracked.

factualcontestednovelty 3/4durability 2/4· Leopold Aschenbrenner

“Common Crawlonline is about 30 trillion tokens.Llama-3was trained on 15 trillion tokens. We're already using all of the data.”

0.45

A current alignment advantage is that we can read what models are thinking via legible chain-of-thought (we bootstrap by pre-training them to think in English), so initial automated alignment researchers may not need complicated mechanistic interpretability; but this is likely not the most efficient architecture—a recurrent architecture with internal states would be more efficient—so during a rapid intelligence explosion we risk transitioning within a year from legible, understood systems to alien, illegible architectures, making it an incredibly hairy period for alignment.

causalcontestednovelty 3/4durability 2/4· Leopold Aschenbrenner

“for the initial automated alignment researchers, we don't need to do any complicatedmechanistic interpretability. You can just read what they're thinking, which is a huge advantage. However, it's very likely not the most efficient way to do it.”

0.45

In explosive-growth AI worlds, real interest rates should rise a lot: on the demand side, firms borrow like crazy for clusters and robo-factories seeking high ROI, and on the consumer-saving side, the Euler equation implies that if people expect higher real growth rates they're less willing to give up consumption today, pushing rates up—so at some point there's a 'big bond short,' and higher growth expectations actually push equities down because the interest-rate effect outweighs the growth-rate effect.

forecastcontestednovelty 3/4durability 2/4· Leopold Aschenbrenner

“At some point, actually, there's going to be real headwind to equities from real interest rates. In these sorts of explosive growth worlds, you would expect real interest rates to go up a lot.”

0.45

The period after the intelligence explosion but before the industrial/robot explosion is extremely volatile and dangerous: an adversary who is behind has an enormous incentive for a first strike to take out the leader's data center before they gain a decisive lead, so data centers will likely have to be protected with the threat of nuclear retaliation—analogous to Eisenhower and Kennedy threatening nuclear war over West Berlin.

forecastfringenovelty 4/4durability 2/4· Leopold Aschenbrenner

“There's actually an enormous incentive for first strike, if they can take out your data center. They know you're about to have this command, a decisive lead.”

0.45

People at SF parties vastly underrate the chance that AGI development becomes a government project; when there is literal superintelligence on the cluster—a billion superintelligent scientists who can hack everything, Stuxnet Chinese data centers, and build robo-armies—the national security state will inevitably get involved, whether through nationalization, public-private partnership, a defense-contractor relationship, or a government project absorbing the people.

forecastfringenovelty 4/4durability 2/4· Leopold Aschenbrenner

“people vastly underrate the chances of this looking like a government project.”

0.44

High-fertility religious subgroups are not a scalable solution to fertility decline: even Mormons' fertility rates are dropping, and once such groups (Mormons, Amish) grow big enough to come into close contact with normal society, they become normalized and their fertility rates fall (e.g., from ~4 to ~2 over 10-20 years) under the overwhelming force of modernity.

causalcontestednovelty 2/4durability 3/4· Leopold Aschenbrenner

“Once these religious subgroups that have high fertility rates grow big enough, they become too close in contact with normal society and become normalized. Their fertility rates drop from maybe like four to two in the course of 10-20 years.”

0.44

The development of nuclear weapons, though regretted by many Manhattan Project scientists, was inevitable given a world war and cold war, and actually went well: it's physically impossible to have pocket nukes for everyone, so the US led, built a new world order with a few great powers and a non-proliferation regime trading no-military-use for civilian-tech help, and enforced safety norms—an outcome that could have gone much worse and serves as a template for managing AGI.

factualcontestednovelty 2/4durability 3/4· Leopold Aschenbrenner

“nukes went really well. That could have gone terribly. It's not physically possible to have something like pocket nukes for everybody, where WMDs proliferated and were fully democratized.”

0.44

The misconception that AGI development will be a beautiful decentralized community of collaborating coders is false: it requires a $100 billion to trillion-dollar cluster that few will have, open source currently only works because algorithms were published (and that's stopping), and because Moore's Law is slow the $100 billion computer won't cost $1,000 within your lifetime—so the private world will be just two or three big players.

factualcontestednovelty 2/4durability 3/4· Leopold Aschenbrenner

“There's a misconception that AGI development will be a beautiful, decentralized thing, a giddy community of coders collaborating. That's not how it's going to look. It's a $100 billion or trillion-dollar cluster. Not many people will have it.”

0.44

Inputs to research genuinely matter and magnitudes are decisive: even if Patrick Collison were dictator of Switzerland or Liechtenstein with ideal institutions but a fixed talent pool, that country still couldn't outcompete the United States in scientific and technological progress—the US produces vastly more progress than tiny nations because of scale, contradicting the view that institutions alone determine output.

causalcontestednovelty 2/4durability 3/4· Leopold Aschenbrenner

“Even then, even if Patrick Collison were the dictator, Switzerland still wouldn't be able to outcompete the United States in scientific and technological progress. Magnitudes matter.”

0.44

Unlike most recent Silicon Valley products, AI is fundamentally an industrial process: each new frontier model requires building giant new compute clusters, new power plants, and soon new fabs, not just code.

factualcontestednovelty 2/4durability 3/4· Leopold Aschenbrenner

“Unlike most things that have recently come out of Silicon Valley, AI is an industrial process. The next model doesn't just require some code. It's building a giant new cluster.”

0.44

RLHF, while a successful technique for controlling current models, probably won't scale to superhuman systems because it relies on human raters giving feedback, but superintelligent models will produce outputs too complex for humans to evaluate—millions or billions of lines of complicated code where you can't tell if it's hacking, exfiltrating itself, or going for the nukes—so thumbs-up/thumbs-down RLHF stops working and a successor technique is needed (a hard but solvable problem).

factualcontestednovelty 2/4durability 3/4· Leopold Aschenbrenner

“RLHF probably won't scale to superhuman systems. RLHF relies on human raters giving feedback, but superintelligent models will produce complex outputs beyond human comprehension.”

0.44

Counterintuitively, the much harsher peace imposed on Germany after WWII worked out better than the milder Peace of Versailles after WWI: WWII destruction was total (over half the housing stock destroyed in major cities, ~40% of men dead in some birth cohorts, ~20 million displaced, smaller borders, a completely imposed new political system), yet unlike the post-WWI resurgence of German nationalism, this produced a stable, peaceful, allied Germany—suggesting the value of decisively imposing a new order.

causalcontestednovelty 2/4durability 3/4· Leopold Aschenbrenner

“The peace imposed after World War Two was much more strict. The whole country was destroyed... But in some sense, that worked out better than the post-World War I peace where there was this resurgence of German nationalism.”

0.44

The dysfunctional US immigration system nearly forced Dwarkesh into a default 'code monkey' path: as a dependent on his doctor father's H-1B, he faced being kicked off the decades-long Indian green card queue at 21 and restarting, unable to do a startup; only because COVID fast-tracked the backlog (fewer incoming foreigners) did he get a green card months before turning 21, which contingently enabled him to pursue the podcast—illustrating the large cultural and human-capital impact of immigration policy.

factualestablishednovelty 2/4durability 2/4· Dwarkesh Patel

“When you're 21 you get kicked off the queue and you have to restart the process. My dad's a doctor and I'm on his H-1B as a dependent.”

0.43

Secrecy can be decisive in technology races: early incorrect graphite-moderator measurements led Germany down the wrong heavy-water path, but Enrico Fermi's new measurements showed graphite would work; Leo Szilard persuaded a reluctant Fermi not to publish, which (just in time) prevented the Nazis from learning graphite worked and kept them on the wrong path—a key reason the German atomic project failed and fell far behind.

causalcontestednovelty 3/4durability 3/4· Leopold Aschenbrenner

“Fermi not publishing meant that the Nazis didn't figure out graphite would work. They went down the path of heavy water, which was the wrong path. This is a key reason whythe German projectdidn't work out.”

0.43

Alignment is fundamentally dual-use: it's just the underlying technique for getting AIs to do whatever you want, so it enables 'CCP bots' that take the red flag of Mao across the galaxies just as much as it enables US AIs to follow the Constitution, disobey unlawful orders, and respect checks and balances—meaning solving alignment circumscribes the AI-takeover branches of the future and accentuates conflict between humans over which visions to instantiate.

factualcontestednovelty 3/4durability 3/4· Leopold Aschenbrenner

“alignment is often dual use... alignment enables the CCP bots. Alignment also is what you need to get the US AIs to follow the Constitution, disobey unlawful orders, and respect separation of powers.”

0.43

The observation that ideas get harder to find as research effort rises is just an endogenous equilibrium, not a suspicious coincidence: plotting log cumulative research effort against log output (GDP, OOMs of algorithmic progress, transistors per square inch, solar cost) yields a straight line because the difficulty of finding new ideas depends on how much progress has already been made—analogous to supply equaling demand in a market clearing.

definitioncontestednovelty 3/4durability 3/4· Leopold Aschenbrenner

“Isn't it a coincidence that supply equals demand and the market clears? It's the same here. The difficulty of finding new ideas depends on how much progress has been made.”

0.42

The intelligence explosion starts with AI research specifically because, unlike plowing fields or digging coal, AI research doesn't run into real-world physical bottlenecks—it's all virtual code with clear metrics that AI labs deeply understand—so even if the broader economy faces bottlenecks, AI research can accelerate first, then translate quantity into qualitatively superhuman systems that solve the bottlenecks (e.g., building robots to flip burgers).

causalcontestednovelty 3/4durability 2/4· Leopold Aschenbrenner

“AI research doesn't run into these real-world bottlenecks. It doesn't require plowing a field or digging up coal. It's just doing AI research.”

0.41

Situational awareness isn't a one-time realization but a continuous process requiring mental flexibility and willingness to change your mind: many have had their brains broken in the AGI debate by calcifying into ideologies—doomers prescient a decade ago but failing to update on deep learning's empirical realities, e/accs who shitpost without thinking it through, stagnationists who think it's just chatbots—so the most important thing is that good people take the issues seriously, keep updating, and face reality head-on, because no one else is reliably 'on it'.

normativespeaker onlynovelty 3/4durability 3/4· Leopold Aschenbrenner

“There's a sort of mental flexibility and willingness to change your mind that's really important. This is how a lot of brains have been broken in the AGI debate.”

0.41

Taiwan-watcher circles consider the late 2020s the maximum period of risk for Taiwan because military modernization cycles and extreme fiscal tightening on the US military budget over the past decade or two have put US naval capacity in a trough by then—coinciding with when China says it wants to be ready—and Taiwan's centrality to the chip supply chain makes it analogous to all the world's uranium being in Berlin.

factualcontestednovelty 3/4durability 1/4· Leopold Aschenbrenner

“people often talk about the late 2020s as themaximum period of risk for Taiwan. Military modernization cycles and extreme fiscal tightening on the US military budget over the last decade or two have meant that we're in a trough by the late twenties in terms of overall naval capacity.”

0.41

AGI is not priced into financial markets despite the efficient-market hypothesis because a small group of smart people (like those in San Francisco) have alpha over the rest of society in seeing the future—just as a similar group correctly called COVID and shorted the market; markets haven't priced AGI for the same reason the government hasn't nationalized the labs: society simply hasn't priced it in or had the idea diffuse, and not many people take it seriously.

factualcontestednovelty 3/4durability 1/4· Leopold Aschenbrenner

“I used to believe in theEMHguy as an economist. But now, I think there are groups of smart people, like those in San Francisco, who have alpha over the rest of society in seeing the future.”

0.40

The intelligence explosion starts narrowly with cognitive jobs—the highest ROI being to make the AI better and solve robotics—then expands: glasses-and-AirPods-equipped workers become skilled technicians instantly, robots arrive, and R&D expands into biology and other fields, making it a gradual but rapid cascading process rather than an instantaneous jump.

forecastfringenovelty 3/4durability 2/4· Leopold Aschenbrenner

“It's a little bit more gradual. It's an explosion that starts narrowly. It can do cognitive jobs. The highest ROI use for cognitive jobs is to make the AI better and solve robotics.”

0.40

Responsible scaling policies and current safety regulations are important not as a final solution but for helping figure out which world we're in and flashing warning signs when we're close to dangerous capabilities; they preserve optionality—letting us see how things go while being prepared to act when red lights start flashing, such as when the automated AI researcher arrives and it becomes crunch time.

normativecontestednovelty 2/4durability 2/4· Leopold Aschenbrenner

“They're important for helping us figure out what world we're in and flashing the warning signs when we're close.”

0.40

The government, unlike private companies, has actual decades of experience handling nukes and powerful technology and genuinely cares about this stuff through its national security state; the AI labs' first stress test (the November board events) went badly, and in a private world rogue employees could use superintelligences to do whatever they want, so government checks and balances have a stronger track record than corporate governance for this class of technology.

factualcontestednovelty 2/4durability 2/4· Leopold Aschenbrenner

“The government has actual decades of experience and actually cares about this stuff. They deal with nukes and really powerful technology.”

0.40

Building trillion-dollar clusters in the US is feasible without displacing existing power: the 10GW cluster needs only a few percent of US natural gas production, and the 100GW cluster (over 20% of US electricity) is doable either by embracing natural gas (production has nearly doubled in a decade) in West Texas or the Marcellus Shale, or via a green-energy megaproject path requiring broad deregulation (FERC reform, blanket NEPA exemptions, permitting reform)—ideally both.

factualcontestednovelty 2/4durability 2/4· Leopold Aschenbrenner

“There are basically two paths to doing it in the US. One is you just have to be willing to do natural gas. There's ample natural gas.”

0.40

China should not be counted out despite export controls: they can already make 7-nanometer chips and may mature the ability to make many, they have vast latent industrial capacity, they can build power fast (adding as much power in a decade as an entire US grid), and once the CCP wakes up to AGI it could outbuild the US—which is why China might literally win the race if it can steal the weights and algorithms.

factualcontestednovelty 2/4durability 2/4· Leopold Aschenbrenner

“There's a lot of latent industrial capacity in China. They are able to build a lot of power fast. Maybe that isn't activated for AI yet.”

0.40

The argument that AI labs need Middle Eastern money is weak because America has plenty of capital—trillion-dollar companies and very deep financial markets; Microsoft could issue hundreds of billions in bonds to pay for clusters, so the only reason Gulf dictators are getting a seat at the AGI table is that labs are actively courting them, not because their capital is uniquely necessary.

factualcontestednovelty 2/4durability 2/4· Leopold Aschenbrenner

“The Middle East does offer capital, but America has plenty of capital. We have trillion-dollar companies... Microsoft could issue hundreds of billions of dollars of bonds and they can pay for these clusters.”

0.40

A Tyler Cowen-esque blind spot for people in SF AI circles is that they're not exposed to how a normal person in America will (or won't) use AI; the difficulty of a typical engineer at a normal company integrating AI will be a major bottleneck to diffusion, which means Aschenbrenner may be overrating near-term revenue by assuming everyone will adopt it readily.

factualcontestednovelty 2/4durability 2/4· Leopold Aschenbrenner

“You're not exposed to how a normal person in America will use AI. That kind of thing will be a bottleneck to the diffusion of these things. I'm overrating the revenue because I assume everyone is adopting it.”

0.40

The right strategy is to first achieve a clearly winning position (locked-down secrets, 100GW US cluster, natural gas and deregulation) so it is clear the democratic coalition will win, then offer China a stabilizing Atoms-for-Peace deal: respect them, don't use superintelligence against them, give them a slice of the galaxy, benefit-share, and enforce a compute ratio agreement with opposing AIs—creating a US-led, democracy-led new world order that avoids a volatile WMD arms race to the death.

normativespeaker onlynovelty 4/4durability 2/4· Leopold Aschenbrenner

“When it is clear that the democratic coalition is well ahead, you go to China and offer them a deal.”

0.38

Frontier model inference costs may not rise much despite huge capability gains: GPT-3 at launch was actually more expensive than GPT-4 now, so inference cost stayed roughly constant over vast capability gains; theoretically, Chinchilla scaling allocates half of added compute to bigger models and half to more data, so 0.5 OOM/year of algorithmic progress saves exactly enough to compensate for making the model bigger—meaning hundreds of millions of AI-researcher instances are affordable.

factualcontestednovelty 3/4durability 2/4· Leopold Aschenbrenner

“GPT-3 when it launched was actually more expensive than GPT-4 now. Over vast increases in capability gains, inference cost has remained constant.”

0.38

If the US locks down the labs and leverages its better scientists, the lead over China would be two years, which makes all the difference; even six months to a year matters enormously because during the intelligence explosion a year could be the difference between a human-level and a vastly superhuman system (perhaps five OOMs), and a buffer is one of the most important inputs to whether we destroy ourselves or get alignment right.

causalfringenovelty 3/4durability 3/4· Leopold Aschenbrenner

“A year might be the difference between a system that's sort of human-level and a system that is vastly superhuman. It might be like five OOMs.”

0.38

The dangerous period stems from speed and offense-defense imbalances as the 20th century compresses into years; but if we make it through, the situation can stabilize—for example bioweapons are initially offense-dominant (an attacker can release 1000 synthetic viruses) but eventually a universal defense against every possible virus restores a stable offense-defense balance, after which civilian uses can run free much as planes restricted certain military capabilities while letting civilian aviation flourish.

forecastspeaker onlynovelty 3/4durability 2/4· Leopold Aschenbrenner

“Bioweapons are a huge issue initially. An attacker can create 1000 different synthetic viruses and spread them. It's hard to defend against each. Maybe at some point, you figure out a universal defense against every possible virus, then you're in a stable situation again.”

0.38

It is really important that different political factions and parties each get their own superintelligence even if you disagree with their values, modeled on AIs that believe in the Constitution the way Supreme Court justices and Federal Reserve officials sincerely believe in their principles—so that classical liberal processes of pluralism, debate, and error correction can continue to play out rather than being locked in by a single AI's values.

normativespeaker onlynovelty 3/4durability 2/4· Leopold Aschenbrenner

“It's really important that each political party gets to have their own superintelligence. You might totally disagree with their values, but it's important that they get to have their own kind of superintelligence.”

0.36

Scaling laws have held over roughly 15 orders of magnitude: concatenating the original Kaplan scaling laws paper (10^-9 to 10 petaflop days) with additional compute up to GPT-4 (assuming some algorithmic progress) shows the relationship has held over probably 15+ OOMs—an extraordinarily robust empirical regularity, though it holds for the next-token-prediction loss and the data wall is a headwind requiring a new paradigm.

factualestablishednovelty 2/4durability 2/4· Leopold Aschenbrenner

“If you look at theoriginal Kaplan scaling laws paper— it went from 10^-9 to 10 petaflop days — and then concatenate additional compute from there to GPT-4, assuming some algorithmic progress, the scaling laws have held probably over 15 OOMs.”

0.36

Aschenbrenner reports hearing from multiple people (not from his time at OpenAI, and without seeing the memo) that several years ago OpenAI leadership laid out a plan to fund and sell AGI by starting a bidding war between the governments of the United States, China, and Russia—suggesting a willingness to sell AGI to authoritarian governments and play them off each other.

factualspeaker onlynovelty 4/4durability 1/4· Leopold Aschenbrenner

“at some point several years ago, OpenAI leadership had laid out a plan to fund and sell AGI by starting a bidding war between the governments of the United States, China, and Russia.”

0.36

Aschenbrenner states he was fired from OpenAI ostensibly for 'leaking' (sharing a six-month-old preparedness brainstorming document with three external researchers for feedback, which was normal practice), but that leadership explicitly told him the reason it was a firing rather than a warning was an HR warning he'd received for sharing a memo—arguing OpenAI security was egregiously insufficient against weight/algorithm theft by foreign actors—with the board, after which an HR person told him worrying about CCP espionage was racist.

factualspeaker onlynovelty 4/4durability 1/4· Leopold Aschenbrenner

“Last year, I wrote an internal memo about OpenAI's security, which I thought was egregiously insufficient to protect against the theft of model weights or key algorithmic secrets from foreign actors.”

0.35

Learning has two phases: pre-training is like passive lecturing where words fly by, while real learning happens by reading, thinking, having an internal monologue, attempting practice problems, and failing until it clicks—humans transition from being pre-trained (grade school) to teaching themselves (by college), and models are just now entering that self-learning regime via self-play, synthetic data, and RL.

definitionspeaker onlynovelty 3/4durability 2/4· Leopold Aschenbrenner

“You were pre-trained in grade school and high school. At some point, you transition to being able to learn by yourself... Models are just starting to enter that regime.”

0.35

AI lab employees currently wield enormous power (as shown in the November OpenAI board event), but this power is rapidly depreciating; once AI research itself is automated, the employees will lose their leverage and control will concentrate in a few people commanding armies of automated AIs, alongside politicians, generals, and the national security state.

forecastspeaker onlynovelty 3/4durability 2/4· Leopold Aschenbrenner

“Right now, the AI lab employees have so much power. You saw thisNovember event. It's so much power. Both are going to get automated and they're going to lose all their power.”

0.35

The AGI project will likely look like Operation Warp Speed—a public-private partnership where all the companies are involved, the government makes big advance market commitments and grants permission, and America's deep bench of competent managerial talent is mobilized—rather than a directly military Manhattan Project; merging different labs' codebases and infrastructure (like the Brain-DeepMind merger, which was messy but worked) is feasible.

forecastspeaker onlynovelty 3/4durability 2/4· Leopold Aschenbrenner

“The project will look closer to Operation Warp Speed. You'll have all the companies involved in the government project.”

0.35

Reaching $100 billion a year in AI revenue, needed to justify hundred-billion-dollar clusters, is feasible: selling a $100/month AI add-on to a third of Microsoft Office's ~300 million subscribers would generate $100 billion, and that only requires the AI to deliver a few hours of productivity per knowledge worker per month.

forecastcontestednovelty 2/4durability 1/4· Leopold Aschenbrenner

“There are like 300 million Microsoft Office subscribers. They haveCopilotnow... Suppose you sold some AI add-on for $100/month to a third of Microsoft Office subscribers. That'd be $100 billion right there.”

0.34

A key lesson from the FTX/Sam Bankman-Fried collapse is that people tend to give successful CEOs a pass on their behavior because they're successful; Aschenbrenner knew SBF was extremely risk-taking, narcissistic, and didn't tolerate disagreement well (they fell out when he flagged that flashy biosecurity grants weren't cost-effective)—so it's worth paying attention to people's character, including those you work for, to save yourself pain down the line.

normativespeaker onlynovelty 2/4durability 3/4· Leopold Aschenbrenner

“I, and people in general, had this tendency to give successful CEOs a pass on their behavior because they're successful. You think that's just a successful CEO thing.”

0.34

Growing up as an outsider—as Aschenbrenner did in Germany skipping grades and moving to the US, or as the article on Mormons outside Utah describes—gives people unusual strength later to be willing to say what they think and take stands alone, because they've already experienced being different and learned they don't have to conform to the Overton window.

causalspeaker onlynovelty 2/4durability 3/4· Leopold Aschenbrenner

“Growing up as an outsider gives you unusual strength later on to be willing to say what you think.”

0.32

The political reactions to AI will be cruder and more surprising than complicated technocratic proposals anticipate: Tucker Carlson on Joe Rogan argued that while nuclear weapons are always immoral, it would be immoral NOT to use them on data centers, because Silicon Valley is building superintelligence that could enslave humanity—an illustration of how, like with COVID, left-right reactions will flip and be blunt rather than nuanced.

factualspeaker onlynovelty 3/4durability 1/4· Dwarkesh Patel

“nuclear weapons are always immoral, except when we would use them on data centers. In fact, it would be immoral not to use them on data centers”

0.32

Much of OpenAI's drama stems from cognitive dissonance: genuinely believing they're building AGI (e.g., Sam raising $7 trillion for chips) while not taking the implications seriously and repeatedly making commitments they don't keep—most notably the public 20% compute commitment to superalignment used to recruit people, which was abandoned because there was always something more urgent than long-term safety research, and security being called the number-one priority but never resourced when trade-offs arose.

causalspeaker onlynovelty 3/4durability 1/4· Leopold Aschenbrenner

“What gets people is the cognitive dissonance between believing in AGI and not taking some of the other implications seriously.”

0.32

OpenAI conditioned access to former employees' vested equity on signing an NDA that prevents making statements about AGI and OpenAI; Aschenbrenner declined to sign even though it cost him close to a million dollars in equity ('freedom is priceless'), implying that current employees who signed could not give the kind of candid podcast he was giving.

factualspeaker onlynovelty 3/4durability 1/4· Leopold Aschenbrenner

“They still offered me the equity, but I didn't want to sign. Freedom is priceless”

0.32

Aschenbrenner is launching an investment firm—a brain trust/think-tank-inside-a-hedge-fund—that bets on AGI and superintelligence arriving before the decade is out, on the logic that the decade after the intelligence explosion is the wildest and most important period, capital will really matter (if AGI were priced in tomorrow you could make 100x or more due to sequencing), and the firm provides freedom to be a public voice of reason with the best situational awareness in the business.

normativespeaker onlynovelty 3/4durability 1/4· Leopold Aschenbrenner

“If AGI were priced in tomorrow, you could maybe make 100x. Probably you can make even way more than that because of the sequencing and capital matters.”

0.31

German culture suffers from a 'tall poppy syndrome' where curious students wanting to learn more are resented and crushed by teachers rather than encouraged, there are no elite undergraduate universities, meritocracy was crushed at some point, and there's pervasive complacency—a backlash against anything elite after WWII (possibly a counter-reaction to Nazi Aryan-race elitism)—contrasting with America's appreciation of excellence.

factualspeaker onlynovelty 2/4durability 2/4· Leopold Aschenbrenner

“In Germany, there's atall poppy syndrome. If you're the curious kid in class wanting to learn more, instead of the teacher encouraging you, they resent you and try to crush you.”

0.31

The best economics produces a crisp, intuitive mechanism that explains the world after you do the hard work to uncover it (his 100-page paper's core takeaway can be explained in 30 seconds), but modern econ academia has become decadent—200-page empirical analyses of narrow questions or theory with gazillions of calibrated parameters spitting out results you can't understand—either because ideas are getting harder to find or the beautiful simple things have been discovered.

normativespeaker onlynovelty 2/4durability 2/4· Leopold Aschenbrenner

“The paper I wrote is long, 100 pages of math, but the core takeaway can be explained in 30 seconds and it makes sense and you don't really need the math. The best pieces of economics are like that.”

0.31

The national security states of both the US and China will not initially recognize AI's decisiveness, but as trend lines become clear over a few more '2023-like' years, there will be a delayed, COVID-style awakening—where most of the world ignores the exponential until a sudden March-2020-style moment triggers radical, crude reactions and intense international competition.

forecastspeaker onlynovelty 2/4durability 2/4· Leopold Aschenbrenner

“You will see some amount of the COVID dynamic. COVID in February of 2020 honestly feels a lot like today.”

0.31

China's internal state of mind and political debate remain strikingly impenetrable to outsiders despite near-total globalization, a globalized internet, and near-perfect translation machines: one can sense the political mood of the UK or Germany from afar, but not that of an average Chinese person or elite—interpreting party documents requires specialized ability to parse specific Mandarin connotations—which Aschenbrenner finds worrying given the stakes of US-China AI competition.

factualspeaker onlynovelty 2/4durability 2/4· Leopold Aschenbrenner

“I really don't feel like I have a sense of what is the state of mind, what is the state of political debate, of an average Chinese person or an average Chinese elite.”

0.30

Alignment during the intelligence explosion cannot be a years-long NIST-style bureaucratic process of expert consensus; it is more like running a war with a fog of war—deciding whether it's safe to do the next OOM when you're three OOMs in and don't fully understand what's happening, generalization curves look bad, and automated researchers' assurances aren't fully trusted—so it relies on a sane chain of command and a safety buffer rather than a deliberative regulatory scheme.

factualspeaker onlynovelty 3/4durability 2/4· Leopold Aschenbrenner

“Alignment during the intelligence explosion is not a years-long bureaucratic process. It's more like a war, with a fog of war.”

0.28

For certain kinds of work, moments of peak productivity matter far more than average productivity: there are periods of effervescence where work flows and other times of background computation; this is why some famous CEOs and founders have been bipolar or manic—the call option on your productivity is the most important thing, obtained by increasing volatility.

factualspeaker onlynovelty 2/4durability 2/4· Leopold Aschenbrenner

“moments of peak productivity matter more than average productivity, at least for the way at work. I have periods of a couple months where there's effervescence and other times, I'm computing stuff in the background.”