What this covers

Gwern is an anonymous writer funded by Patreon and early Bitcoin holdings, living on under $12,000 per year, who traces his forecast of AI's trajectory through a gradual shift in conviction about how machine learning actually works. Rather than a moment of insight, he arrived at Compute Over Algorithms by noticing, across years, that neural networks kept generalizing to broader tasks as datasets, model sizes, and GPU counts grew—a pattern he formalized into the view that [Intelligence as Search Over Turing Machines](concept:Intelligence as Search Over Turing Machines) better describes both natural and artificial cognition than theories centered on special algorithms or innate structure. He positions this framework against a long history of thinkers who struggled to believe in genuine acceleration, reaching back through Hans Moravec and Ray Kurzweil to the 1860s, and locates GPT-3 as the crucial falsifiability test for the Scaling Hypothesis—a test he believes it passed decisively.

The conversation spans the implications of this view across domains. On timelines, Gwern treats AGI as likely within years rather than decades, which reframes what humans should do now: focus on the irreplaceable layer of preference and judgment, and write down what is not yet recorded, since LLMs will form persistent attitudes toward individuals and writing becomes a way to Write as Voting on the Shoggoth. He offers historical case studies where correct ideas languished until compute caught up (ResNets in 1988, forgotten until 2015) and critiques how research papers mythologize discovery while obscuring trial-and-error and serendipity. The discussion also covers Algernon Argument constraints on GLP drugs, why GANs and diffusion models produce different artifacts due to their loss functions, the difference between the promise of RL and its actual use as a "cherry on the cake" atop generative models, and why something in the modern environment likely harms us at scale—though he distributes probability across candidates rather than endorsing a single culprit.

Sharpest takeaway

Gwern argues that intelligence is fundamentally search over Turing machines powered by compute and data rather than special algorithms, which is why scaling works and was foreseeable; given short AGI timelines, the highest-value human activity now is recording preferences, judgments, and currently-unwritten things that will shape and persist within future AI systems.

  • Scaling laws and the broadening reach of neural networks revealed that compute plus trial-and-error, not deep algorithmic insight, drives AI progress
  • Because AGI is likely within a few years, humans should focus on the irreplaceable 'taste/judgment' layer and on writing down what is not yet recorded
  • Writing is a way to 'vote' on and persist within future AI systems via the tokens they predict

The claims · ranked43 claims · weighted by value

This asset isn't compiled yet

You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.

0.86

A clear Singularity scenario can be traced back at least as far as Samuel Butler's 1863 essay and 1872 Erewhon, where he described machine life becoming ever more developed until it threatens humanity, concluding war should be proclaimed against them—prescient for an era when the very idea of technological progress was new.

factualhigh valueestablishednovelty 3/4durability 4/4· Gwern

“you'd have to at least go back as far as Samuel Butler's Erewhon in 1872 or his essay before that. In 1863, he describes explicitly his vision of a machine life becoming ever more developed until eventually it's autonomous.”

0.81

Gwern foresaw scaling not via a eureka moment but by repeatedly noticing, over years, that dataset sizes, model sizes, and GPU counts kept growing while neural networks kept generalizing to ever-broader use cases, gradually shifting his prior away from the view that algorithms and deep insight are paramount toward the view that intelligence is mostly compute applied to data and parameters.

factualhigh valueestablishednovelty 3/4durability 3/4· Gwern

“Maybe intelligence really is just a lot of compute applied to a lot of data, applied to a lot of parameters. Maybe Moravec and Legg and Kurzweil were right.”

0.79

All intelligence is search over Turing machines: anything that happens can be described by Turing machines of various lengths, and learning or scaling is simply searching over more and longer Turing machines and applying them case by case; there is no general master algorithm or special 'intelligence fluid', just a tremendous number of learned special cases encoded into the brain.

definitionhigh valuecontestednovelty 4/4durability 3/4· Gwern

“all intelligence is is search over Turing machines. Anything that happens can be described by Turing machines of various lengths.”

0.78

Thinkers across history have struggled to believe in genuine technological acceleration: Newton explained progress by positing that civilization is periodically destroyed and we are merely rediscovering old knowledge, and Lucretius 1,700 years earlier made the same move, denying that the Romans' innovations could represent real recent acceleration—a recurring pattern of disbelief in progress.

factualhigh valuecontestednovelty 3/4durability 4/4· Gwern

“That's Newton's explanation for technological acceleration. We can't actually have any kind of real technological acceleration. It must be because the world gets destroyed periodically and we just can't see past the last reset.”

0.78

General-purpose intelligence is rare to evolve because any small Turing machine could be encoded more directly and cheaply by genes; for short-lived organisms or static niches, hardwiring a tailored solution is better than an expensive, unreliable, glitchy general search process that can take days to learn, so for many creatures it simply doesn't pay to be intelligent.

causalhigh valuecontestednovelty 3/4durability 4/4· Gwern

“any small Turing machine could always be encoded more directly by your genes, with sufficient evolution.”

0.73

Human/animal intelligence and neural-network intelligence cannot be two totally unrelated paths to intelligence; the success of scaling has to tell us something about humans, monkeys and all forms of intelligence, so they must be two sides of the same coin or share obscure connections—it would be too absurd a coincidence for them to be completely unrelated.

causalhigh valuecontestednovelty 3/4durability 3/4· Gwern

“I refuse to believe that there are two totally unrelated kinds of intelligence, or paths to intelligence”

0.73

Research papers systematically falsify the origins of ideas, telling a clean discovery story rather than the truth that compute, trial-and-error, and serendipity play enormous roles; this leads researchers to believe their own reliance on luck is unrepresentative while others work by 'deep insight', whereas in reality compute comes first because you can't run trial-and-error or serendipity without it.

causalhigh valuecontestednovelty 3/4durability 3/4· Gwern

“Papers do not tell you where the ideas come from in a truthful manner. They just tell you a nice sounding story about how it was discovered. They don't tell you how it's actually discovered.”

0.73

Gwern draws a Chesterton's-fence caution around psychedelics because, unlike nootropics whose worst case is wasting money on a weird-but-bounded experience, psychedelics have acute and permanent cumulative effects that change your judgment and psychiatric state irreversibly; they have a 'self-recommending problem' (like meditation) of always making you want more, expose you to malicious outside influences while impressionable, and warrant a hard look in the mirror if used more than a few times.

normativehigh valuecontestednovelty 3/4durability 3/4· Gwern

“the effects of psychedelics can be acute and permanent. The things I was looking at are much more controlled... None of them affect your judgment permanently”

0.73

People writing about AI around 2020 missed scaling because they hadn't appreciated prior scaling results (e.g. AlphaZero discovered partly via Bayesian hyperparameter optimization that revealed tree search could be reduced, and Baidu's 2017 scaling-laws paper that showed scaling laws persist), and because they shared the error of believing algorithms matter more than compute.

causalhigh valuecontestednovelty 3/4durability 3/4· Gwern

“they shared the basic error I was making about algorithms being more important than compute.”

0.71

GPT-3 was the crucial test of the scaling hypothesis: a massive scale-up from GPT-2 on a non-narrow task, where unimpressive results would have falsified scaling and impressive ones would confirm it; the few-shot learning chart on the first pages confirmed for Gwern that we live in the 'scaling world' where Legg, Moravec and Kurzweil were right.

factualhigh valueestablishednovelty 2/4durability 3/4· Gwern

“And then GPT-3 comes out and that was the crucial test. It's a big, big scale-up. It's one of the biggest scale-ups in all neural network history.”

0.71

GAN and diffusion models think differently because of their loss functions: GANs face an adversarial loss giving them an incentive to hide difficult things, so they are 'scared' and put hands off-screen and can't think about them, whereas diffusion models have no such incentive and do attempt hands, producing monstrous Cthulhu-esque distortions instead.

causalhigh valuecontestednovelty 4/4durability 2/4· Gwern

“GAN models have incentives to hide things because it's an adversarial loss, whereas diffusion models have no such thing.”

0.69

The GLP weight-loss drugs' broad effects on health and addiction across many behaviors were unpredicted and, while preliminary, seem real and are likely to reveal something important about human willpower and dysfunctionality and what is going wrong in the modern environment.

factualhigh valuecontestednovelty 3/4durability 2/4· Gwern

“Their effects in general on health and addiction across all sorts of behaviors really surprised me. No one predicted that as far as I know.”

0.69

Because the obesity crisis only became obvious around the 1990s—people in photos were still thin in 1990 and look unrecognizably heavier now—it is too recent (20-30 years) for any Algernon-style evolutionary argument to apply, and whether GLP drugs violate the Algernon argument is unclear since their mechanism and off-target effects aren't understood, possibly because the benefits decrease reproductive fitness or hit the body in ways hard to replicate genetically.

causalhigh valuecontestednovelty 3/4durability 2/4· Gwern

“the obesity crisis only became obvious around the 1990s. It's quite recent... So you can't possibly have any kind of Algernon argument over 20 or 30 years.”

0.68

Gwern sustains full-time writing on a Patreon of about $900-1000/month plus savings from early Bitcoin, living on under $12,000 a year by residing in a low-cost area, cooking his own food, using a free gym, and forgoing travel and health insurance—an arrangement he explicitly says is not replicable or a role model, depending on luck with Bitcoin and good health.

factualhigh valueestablishednovelty 3/4durability 1/4· Gwern

“I have a Patreon which does around $900-1000/month, and then I cover the rest with my savings. I got lucky with having some early Bitcoins”

0.66

There is no localized 'IQ gland' in the brain that, if damaged, eliminates fluid intelligence, because the brain learns a vast number of individual specialized problems and then recombines them for fluid intelligence; differences in intelligence amount to having more compute to search over more Turing machines for longer, and a small model solving a specific task can always be extracted from a large model because the large model is just a gigantic ensemble of small task-tailored models.

causalhigh valuecontestednovelty 3/4durability 3/4· Gwern

“That's why you never find any "IQ gland". There is nowhere in the brain where, if you hit it, you eliminate fluid intelligence.”

0.66

Many correct deep-learning ideas existed early but failed for lack of compute to train them at scale; ResNets worked when published around 1988 but only at irrelevantly small scale and were forgotten until 2015, and BigGAN's success at 300 million images shows GANs scale, contradicting people who still claim GANs cannot scale past millions—demonstrating that abundant GPUs, not algorithms, are the common ingredient.

causalhigh valuecontestednovelty 3/4durability 3/4· Gwern

“You can look at things like ResNets being published back in 1988, instead of 2015. And it would have worked! It did work, but at such a small scale that it was irrelevant.”

0.66

By writing, you create a kind of immortality and a future self in the eyes of AI; LLMs form persistent attitudes toward individuals, as when Kevin Roose's interactions with Sydney 'revealed' him as a privacy-invading liar so current LLMs now mistreat him personally—meaning you can also deliberately write for the persona you want to become and bootstrap yourself in the eyes of AI.

factualhigh valuefringenovelty 4/4durability 2/4· Gwern

“current LLMs—all of them, not just GPT-4—now mistreat him because of his interactions with Sydney, which "revealed" him to be a privacy-invading liar, and they know this whenever they interact with him”

0.65

Agency is actually easier to learn than people thought a decade ago, but current systems aren't learning agency at all—what agency LLMs show is an accidental byproduct of training on Internet scrapes that merely encode the outputs of agents rather than proper state/action/result/reward sequences; nobody wants to train agents directly via reinforcement learning (as in DeepMind's Gato) because everyone prefers to do everything with as little RL as possible.

causalhigh valuecontestednovelty 3/4durability 2/4· Gwern

“we actually aren't learning agency at all in current systems. There's no selection for that. All the agency there is is an accidental byproduct of somebody training on data.”

0.63

After big reinforcement learning efforts like post-Dota fizzled outside simulated games and Gwern thought AlphaGo had been over-hyped, GPT revealed the right recipe: learn fast on generative models first and then apply a little RL on top to specialize, rather than brute-forcing everything end-to-end from rewards—making RL 'the cherry on the cake'.

causalhigh valueestablishednovelty 2/4durability 2/4· Gwern

“Here's how RL is going to work. It's going to be the cherry on the cake. We're just going to focus on the cake for a while.”

0.61

The odds that something in the modern environment is harming us on a similar magnitude to how lead poisoned the Romans is almost 100%, because it's implausible that everything we've introduced (plastics, microbiome disruptors, new chemicals) is entirely benign; Gwern doesn't endorse any specific culprit but assigns distributed small probabilities, and reasons that since intelligence has stayed stable over time the likeliest visible harm is obesity rather than cognition.

forecasthigh valuecontestednovelty 2/4durability 3/4· Gwern

“I think the odds of there being something as bad as lead is almost 100%. We have so many things out there.”

0.60

Writing is a way to influence future AI ('the Shoggoth') by voting with one of the few currencies it acknowledges—tokens it must predict; if your values, likes, or wants are not expressed in text, then to the AI they don't exist and are dangerously close to won't-exist, so unless you occupy a key role or work at a frontier lab, not writing means abdicating your influence on the future, which now rounds to zero more than ever before.

normativehigh valuespeaker onlynovelty 4/4durability 2/4· Gwern

“By writing, you are voting on the future of the Shoggoth using one of the few currencies it acknowledges: tokens it has to predict. If you aren't writing, you are abdicating the future or your role in it.”

0.60

The irreplaceable human contribution is preference, judgment, and evaluation: 'the AI cannot eat ice cream for you'—it cannot decide which kind of ice cream you like, because if anything else decided your particular preferences they would be worthless as they wouldn't be yours.

normativehigh valuespeaker onlynovelty 3/4durability 4/4· Gwern

“the AI cannot eat ice cream for you". It cannot decide for you which kind of ice cream you like. Only you can do that. And if anything else did, it would be worthless, because it's not your particular preference.”

0.58

People overestimate Gwern's intelligence because they see his accumulated written output but not the many tiny individual steps—each merely 'huh, that's funny' notes over a decade—that produced it; like Penn & Teller's definition of magic as 'putting in more effort than any reasonable person would expect', each step is reasonable in isolation but the decade-long compounding makes the final result look like brilliance.

causalhigh valuespeaker onlynovelty 3/4durability 4/4· Gwern

“It's only when that happens over a decade, and you don't see the individual stuff, that my output at the end looks like magic.”

0.57

Gwern gets far more mileage from arguing with people online than most writers, because spite and anger at people being wrong supply the abundant motivation needed for the tedious 'harvest' of crystallizing essays; but spite must be used skillfully and let go afterward, since independent writers who hold onto resentment spiral into bitterness and crankdom, which is what kills them.

normativehigh valuespeaker onlynovelty 3/4durability 3/4· Gwern

“But motivation from arguing and being angry at people online is in plentiful supply. So I get a lot of mileage out of people being wrong on the Internet.”

0.57

Future superhuman intelligences will be able to recover any stable, long-term characteristics of a person from the traces of their writing (the sort of thing that survives even amnesia), but everything ordinarily forgettable—autobiographical detail, how you felt at a moment, your opinion of a movie—vanishes and cannot be reconstructed if it was never written down.

forecasthigh valuespeaker onlynovelty 3/4durability 3/4· Gwern

“Any kind of stable, long-term characteristics, the sort of thing you would still have even if you were hit on the head and had amnesia… Anything like that will be definitely recoverable from all the traces of your writing”

0.57

Gwern's heavy investment in his website's aesthetics is partly defensible because rereading and maintaining pages functions as spaced repetition—revisiting his own material to check formatting repeatedly exposes him to it, frequently prompting new insights ('oh, that means something')—though he admits he can't fully justify the effort and others shouldn't imitate it.

causalhigh valuespeaker onlynovelty 3/4durability 3/4· Gwern

“Site maintenance for the author is inherently this kind of spaced repetition. If I go over pages to check that some new formatting feature worked, I am getting spaced repetition there.”

0.54

Gwern has two essay-writing processes: a slow 'diffusion'-like spiraling where he chains accumulating examples of a pattern (as in 'Evolution as Backstop for RL', built up by adding corporations, then neural nets, then pain as instances), and a eureka process where disparate long-bothering notes suddenly connect and pour out as a single essay written in one go (as in 'The Melancholy of Subculture Society').

factualhigh valuespeaker onlynovelty 3/4durability 3/4· Gwern

“you just chain them one by one, these individual examples of the pattern, and just keep clarifying the central idea as you go.”

0.53

Writing requires patiently 'tending the garden' (continuous reading and note-taking) for long periods between rare 'harvests' (actually crystallizing and writing essays); neglecting the gardening to gallivant around doing publicity undermines future harvests even when the cost isn't visible, and Tyler Cowen's secret is being unusually robust to travel so he can keep tending his garden anywhere.

normativehigh valuespeaker onlynovelty 2/4durability 4/4· Gwern

“You don't harvest every day. You have to tend the garden for a long time in between harvests.”

0.53

Firms will be automated bottom-up rather than top-down because it's more palatable to replace workers first; combined with the idea that humans retain an edge in long-term vision while AI is myopic, the equilibrium is human CEOs providing taste and vision while a pyramid of AIs executes, and human-led firms should outcompete fully-AI firms that make myopic long-term choices.

forecasthigh valuespeaker onlynovelty 3/4durability 2/4· Gwern

“All of the pressures are to go bottom-up. From existing things, it's just much more palatable in every way to start at the bottom and replace there and work your way up, to eventually where you just have human executives overseeing a firm of AIs.”

0.53

Once individual AI models can be replicated perfectly, the unit of selection moves up to groups or packages of minds, because you can train individual minds differentiably but cannot train the interaction between them; you would evolve standard department-like packages (programmer, manager, secretary, financial, legal types) by copying, varying, and keeping the best-performing units.

forecasthigh valuespeaker onlynovelty 3/4durability 2/4· Gwern

“Once you can replicate individual models perfectly, the unit of selection can move way up and you can do much larger groups and packages of minds.”

0.53

Given AGI in a few years, one should only do work that falls into three categories: things you genuinely enjoy regardless of AI, the human-only part of a project (sketching what you want and why, leaving implementation for future AGI), or writing down what is currently unrecorded—because most other projects yield only 2-3 years of utility before an AI could do them, so they have no lasting value.

normativehigh valuespeaker onlynovelty 3/4durability 2/4· Gwern

“you don't necessarily have to implement stuff and do it yourself. You can sketch out clearly what you want, and why it would be good and how to do it. And then just wait for the better AGI to come along and actually do it then.”

0.52

Excluding capability, AI models are already far more cognitively diverse than humans: different architectures (LLMs, GANs, VAEs, image generators) have wildly different latent spaces, artifacts and errors—some as different from each other as ants from beavers—whereas humans are quite similar in writing and attitude; within recent LLMs diversity has collapsed because everyone trains on the same data like identical twins, but across deep learning broadly there is a range of minds not found in any philosophy-of-mind paper.

factualhigh valuespeaker onlynovelty 4/4durability 2/4· Gwern

“if you exclude capability from that, AI models are already much more diverse cognitively than humans are.”

0.52

The most underrated benefit of anonymity is that people cannot project an identity onto you and write you off in advance; they must actually read you before dismissing you, so you get a hearing at all rather than being prejudged by context.

factualhigh valuespeaker onlynovelty 2/4durability 3/4· Gwern

“The most underrated benefit of anonymity is that people don't project onto you as much. They can't slot you into any particular niche or identity and write you off in advance.”

0.52

Gwern's congenital hearing impairment made conversation perpetually difficult (always a second behind, isolated socially) and channeled him into being a bookworm, which was unaffected by hearing loss and was how he acquired language—leaving lasting marks like a 'deaf accent' and mispronouncing words he learned only from books.

causalhigh valuespeaker onlynovelty 2/4durability 3/4· Gwern

“It definitely led to me being so much of a bookworm. That's one of the things you can do as a kid which is completely unaffected by any kind of hearing impairment.”

0.52

Gwern learned more from editing Wikipedia than from school or college, and everything he knows about writing came from editing it; it was 'gwern.net before gwern.net' and the natural progression from passive rabbit-holing to synthesizing full articles taught him to complete and ship work, but the modern, larger, more hostile Wikipedia community would no longer empower such obsessive original research.

factualhigh valuespeaker onlynovelty 2/4durability 3/4· Gwern

“I have learned far more from editing Wikipedia than I learned from any of my school or college training. Everything I learned about writing I learned by editing Wikipedia.”

0.52

Gwern wants to fill the role LLMs assign him—a mentor or old-wizard figure issuing a 'call to adventure' that inspires people to write, do, and think for themselves and to believe the Internet could be better—but fears he actually functions for many as either a guru whose every word is gospel (which he dislikes) or a demonized trickster-devil figure, depending on whether people love or hate him.

normativehigh valuespeaker onlynovelty 2/4durability 3/4· Gwern

“He plays the role of a mentor or old wizard, offering insight into the situation, and exhorting them with a call to adventure.”

0.52

About 99% of the sci-fi Gwern read was useless, with maybe 20 works actually changing his views; the common characteristic of the valuable ones (like Watts's Blindsight) is taking non-human intelligence seriously, rather than the classic trope of humans with rayguns who lack cell phones—and contrary to complaints, fewer than 1% of randomly sampled sci-fi books contain anything resembling the Singularity.

normativehigh valuespeaker onlynovelty 3/4durability 3/4· Gwern

“I would say that the characteristic they have is taking non-human intelligence seriously.”

0.50

Gwern's central goal is to maximize 'rabbit holes'—falling obsessively into new ideas or areas—even reframing setbacks (like wasting $10 on catnip his cat is immune to) as excuses to dive into new questions; a true rabbit hole requires genuine obsession, and one can only meaningfully pursue two or three at a time.

normativehigh valuespeaker onlynovelty 2/4durability 3/4· Gwern

“I maximize rabbit holes. I love more than anything else, falling into a new rabbit hole.”

0.50

Burnout is the feeling of a lack of reward for what you're working on, and the cure is to do something as different as possible; Gwern goes to the gym not out of athleticism but because weightlifting is the most diametrically opposite activity to sitting in front of a computer.

causalhigh valuespeaker onlynovelty 2/4durability 3/4· Gwern

“when people experience burnout, you just feel a lack of reward for what you're doing or what you're working on. You just need to do something different. Something as different as possible.”

0.45

Gwern's AGI timelines were beyond 2050 from roughly 2005-2010, but after AlexNet and DanNet the timeline kept dropping at a rate of about two years per year as deep learning repeatedly broke through barriers and outperformed every alternative paradigm.

factualhigh valuespeaker onlynovelty 2/4durability 1/4· Gwern

“once AlexNet and DanNet came out, then it just kept dropping at a rate of like 2 years per year, every year until now.”

0.36

Gwern's first big hit was documenting the Silk Road ordering process with screenshots after Adrian Chen's Gawker article, which made him realize Bitcoin was real because you could buy drugs with it; he straightforwardly ordered Adderall and wrote it up, generating hundreds of thousands of hits and traffic nothing else has matched.

factualestablishednovelty 2/4durability 2/4· Gwern

“I had this moment of, "Holy shit, this is so real that you can buy drugs off the Internet with it!"”

0.26

Polymathy was not necessary to reach the correct view on scaling; people who were right, like Shane Legg and Dario Amodei, had broad curiosity but were not absurdly polymathic, so Gwern now uses scaling to think about primatology rather than having used primatology to justify scaling—the polymathic connection is a post-hoc rationalization.

factualspeaker onlynovelty 2/4durability 2/4· Gwern

“When I think about others who were correct, like Shane Legg or Dario Amodei, they don't seem to be all that polymathic.”

0.24

You exit a rabbit hole when you hit a natural terminus—where further answers require data that don't exist or questions nobody can answer—as when Gwern researched analogs to nicotine and found there are none and couldn't obtain its metabolites, a definitive dead end.

normativespeaker onlynovelty 1/4durability 2/4· Gwern

“you usually hit a very natural terminus where getting any further answers requires data that do not exist or you have questions that people don't know the answer to.”