What this covers

Carl Bergstrom, a biologist at the University of Washington, sits down with Sean Carroll to discuss the sources and spread of misinformation in the modern information ecosystem. Drawing on his work tracking information flow through social networks and his course on skepticism in data-driven contexts, Bergstrom identifies a critical shift: while humans have always been skilled at detecting old-fashioned rhetorical manipulation—weasel words, evasion, obvious spin—we are far more vulnerable to claims dressed in the trappings of quantification. The conversation ranges from the evolutionary origins of deception in animals to the specific mechanics of how misinformation travels through social media, with sustained attention to why peer review cannot be our sole defense and why the COVID-19 pandemic became a laboratory for observing Tribal Epistemology at scale.

The episode unpacks several distinct vulnerabilities. Bergstrom explains how numbers stripped of context—a raw count without a denominator or comparator—function as a form of bullshit by design, and how Rates vs Absolute Numbers can be weaponized to mislead while remaining technically true. He frames Old School vs New School Bullshit in terms rooted in Harry Frankfurt's philosophy: the liar knows the truth and hides it, while the bullshitter has no regard for truth whatsoever. He argues that social media has made everyone an editor without the training or incentive structures of professional editors, and that even open science cannot protect itself when facts become tribal markers. The only sustainable defense, Bergstrom contends, is widespread education in quantitative and media literacy—not technological fixes or broad regulation—because game theory and A/B testing show that adversaries will always find new ways around algorithmic barriers.

Sharpest takeaway

Bergstrom argues that the dangerous new form of bullshit is quantitative—claims dressed in the trappings of data, statistics, and algorithms—and that since social media has made everyone an editor, the only viable defense is widespread education in quantitative and media literacy rather than technology or regulation alone.

  • Old-school rhetorical bullshit is easy to detect, but 'new school' bullshit clad in numbers and figures exploits our undertrained trust in quantification.
  • Social media turns everyone into editors without the incentives or training of professional editors, accelerating the spread of misinformation.
  • Politicization (tribal identity) combined with quantitative bullshit and networked spread creates a downward spiral that even open science cannot escape.

The argument · threads7 threads · 28 claims
0.72

Humans uniquely possess language and theory of mind enabling construction of arguments to influence beliefs despite misaligned incentives.

4 pointscentrality 5/5
  • Bullshit is distinct from lying: the liar knows the truth and deliberately leads you away from it, whereas the bullshitter doesn't care what the truth is at all—they use language, figures, or statistical arguments to impress, persuade, or intimidate with blatant disregard for whether the arguments are actually true.

    “the liar knows what she wants you to believe and tries to lead you there away from the truth, and the bullshitter doesn't even care what you... About what the truth is”

  • What is unique in humans is an infinitely expansible combinatorial language allowing an infinite range of meanings, combined with a well-developed theory of mind; together these let us construct arguments and narratives to influence others' beliefs, and because human incentives are rarely perfectly aligned, this creates the setting for widespread lying and bullshitting.

    “we have this sort of infinitely expansible combinatorial language that allows us to create an infinite range of meanings”

  • Deception goes far back in the animal kingdom because manipulating other organisms' access to information is an effective way to manipulate their behavior, as seen in molting stomatopods waving defenseless claws to intimidate rivals and ravens using a theory of mind to fool conspecifics.

    “when they're moulting their claws are just basically a shelled lobster claw, they're defenseless and useless, but they wave them around anyway and try to intimidate each other”

  • The world is awash in bullshit and we are drowning in it.

    “the world's awash in bullshit, and we're drowning in it”

0.68

Bullshit, distinct from lying, uses language and numbers to persuade while disregarding truth, and flourishes especially when cloaked in scientific authority.

3 pointscentrality 5/5
  • Even when numbers are true, there is enormous flexibility in how they are presented, so people can give you true figures yet make you feel completely differently depending on the presentation—for example reporting absolute counts instead of rates to make a quantity seem large, as when '1,000 DACA recipients accused of crimes' sounds alarming while their accusation rate is over an order of magnitude lower than that of citizens.

    “they can give you true numbers, but make you feel completely differently about them, depending on how they're presented”

  • While people are well attuned to 'old school' bullshit (weasel words, paltering, passive-voice evasion), they are far more susceptible to 'new school' bullshit clad in the trappings of science—numbers, statistics, figures, and machine learning—because it is relatively new, the education system hasn't caught up, and we wrongly feel numbers are more objective and real than words.

    “there's this stuff that I call new school bullshit, and that comes clad in the trappings of science, and in particular in the trappings of numbers and statistics and figures and machine learning algorithms”

  • Numbers presented in a way that doesn't allow useful comparisons are almost by their very nature bullshit, because a raw figure with no denominator or comparator (e.g., '123 people had a heart attack while listening to a podcast') prevents meaningful interpretation; this 'quantitative shock and awe' makes it hard to respond on the spot unless one performs a Fermi estimate or quick lookup.

    “When numbers are presented in ways that don't allow you to make useful comparisons, that makes them, almost by their very nature, bullshit”

0.67

Social media algorithms and editorial dynamics amplify extreme content, filter information by tribe rather than truth, and degrade collective decision-making.

6 pointscentrality 4/5
  • The reason we see no evidence of intelligent life out there may not be that civilizations invent nuclear weapons but that they invent social media, which networks people and triggers a spiral of tribal affiliation that degrades collective decision-making and could constitute an existential threat.

    “the answer to the Fermi Paradox... is not that people invent nuclear weapons, it's they invent social media”

  • COVID-19 prompted the most massively open science ever—raw data, preprints, and code shared early on open boards—but because the topic is hyperpoliticized, every paper lands on one side of the political spectrum and gets used as a cudgel, so even when one side's evidence outweighs another's, tribe members won't concede for fear of being seen as belonging to the other tribe.

    “now every single paper that goes out there lands on one end or the other of a political spectrum... that paper gets picked up by one side and used as a cudgel to bash the other side with”

  • Social media has put everyone in the role of editor for one another—filtering and amplifying what others read—but unlike professional editors we lack both the training and the reputation-protecting incentives, and we often share to signal group identity rather than truth, which is why responsible information hygiene requires people to 'think more, share less.'

    “we're all doing the things that professional editors used to do, and one of the big problems is we, not only don't we have the training, but we also don't have the same incentives that professional editors do”

  • Recommendation algorithms like YouTube's are optimized to maximize engagement rather than education or accuracy, and through constant A/B testing they discover that people move toward more extreme content over time, which radicalizes users and pushes them into filter bubbles, as when watching an ISS feed leads to flat-earth video recommendations.

    “all of these algorithms are being... Are optimized to maximize engagement, not education or accuracy or anything else like that”

  • Many aspects of the COVID response—whether the virus is real, whether masks work, whether hydroxychloroquine works—became politicized despite there being no logical reason such questions should fall on the left or the right, illustrating that tribal allegiance, facilitated by social media, can attach to arbitrary factual questions, as shown by people refusing masks purely to signal tribal affiliation.

    “there's going to be a pandemic and some people are going to want hydroxychloroquine. You've got no way to tell me whether that's going to fall on the left or right”

  • Through constant real-time A/B testing on their user interfaces, tech companies effectively know more about specific aspects of human psychology—such as which button colors get clicked and which headline wording works—than the collective discoveries of the academic psychology community, and they know it tailored to each individual user.

    “every single one of these companies know more about psychology than sort of the collective discoveries of the entire psych community, at least about the very specific things about what color buttons do people click on”

0.57

Deception frequency is game-theoretically constrained by repetition and credibility; self-deception is usually error not optimization; media literacy and user control are the sustainable defenses.

7 pointscentrality 3/5
  • There is a game-theoretic optimum mixture of deception: you can bluff some but not always, because if you bluff constantly people will call your bluffs and stop believing you; the frequency of interaction matters, so one can bullshit more in a single appearance than in a repeated relationship with a close colleague.

    “you bluff some, but you can't bluff always and so otherwise, people will call your bluffs all the time”

  • Group selection arguments for traits like maintaining crazy ideas in a population should be treated skeptically without hard quantitative backing, because even if a mixed group outperforms a uniform group, within the mixed group the individuals with reasonable ideas do better, so the genes for crazy ideas are selected away.

    “within that group of people with crazy ideas and reasonable ideas, which ones do better? Well, the ones with reasonable ideas do better”

  • Much self-deception is due to ineffective application of heuristics rather than adaptive optimization; arguments that self-deception is optimal (e.g., to be a better bluffer) clash with basic decision theory, since under most objective functions you do at least as well with correct information, and there is no clear reason evolution couldn't have produced good bluffers who still believe the truth.

    “we're all very good at convincing ourselves of things that are wrong, though I just don't think it's necessarily optimal that we're doing so”

  • By Twyman's law of data analysis, any figure that looks interesting or different is usually wrong—the more unusual or interesting the data, the more likely it resulted from an error of some kind—and this applies to media analysis too, so claims that seem too good or too bad to be true (and which we are most likely to share) are also the most likely to be wrong.

    “Twyman's law for data analysis is that, any figure that looks interesting or different is usually wrong”

  • Regulation should give users control to opt into seeing content (e.g., followed accounts in chronological order) rather than algorithmically manipulated feeds, justified not by bandwidth limits but by the principle—rooted in the FCC fairness doctrine—that an informed electorate is a sine qua non for democracy, while dark targeted political advertising should be banned because it lets messages be micro-targeted invisibly in ways never previously possible.

    “another sine qua non for democracy is to have people able to get access to the information they want, instead of being manipulated by these algorithms”

  • There are three approaches to misinformation on social media—technology (AI detection), regulation, and education—and only education can really be stood on: AI detection will fail against adversarial AI, broad criminalization of misinformation is objectionable on free-speech grounds, leaving media and quantitative literacy as the sustainable solution.

    “the sort of the third leg of the stool that I see there is education. And that's the only one that I think we can really stand on”

  • A key skill in interdisciplinary work is not assuming the other person is an idiot even when they sound like one, because differing conclusions usually trace to hidden differences in assumptions—such as one party reasoning about the expected case while the other reasons about the worst case—and 'poisoning the well' before that difference becomes clear prevents progress.

    “one of the key things to doing good interdisciplinary work is not assuming the other person's an idiot even when they sound like one. Because you're probably not understanding something”

0.54

Evaluating unfamiliar claims requires checking credible expert reception and tracking surprising claims to original sources rather than trusting prestige or presentation.

3 pointscentrality 4/5
  • When evaluating a shocking or extraordinary scientific claim outside your expertise, look at the venues in which it appears and the credence given by well-regarded experts: genuinely world-changing ideas may not come from prominent people, but they will be relatively quickly embraced by a reasonable fraction of serious scholars; if no one credible jumps on it, that itself is informative.

    “the world-changing ideas may not come from the most important, most prominent people, but they will be relatively quickly embraced by some reasonable fraction of serious scholars in that area”

  • Preprint servers radically compress the timeline between putting an idea out and having it adopted and critiqued by the community from about a year to a few days, as Bergstrom found when a physics paper posted to the arXiv drew unsolicited expert feedback within a week, making openness a clear net good for science.

    “within a week, every single person that I wanted to read it had written me back unsolicited and had read the paper and had the comments”

  • Because social media creates a game of telephone—research distilled into a paper, written up in a blog post, picked up by a newspaper, then tweeted—the most important defensive practice is to track surprising claims back to their original source and judge the underlying story for yourself, which demonstrates that despite a bullshit-prone world, truth is out there and reachable.

    “if the tweet seems to good or bad to be true, start tracking back and finding... Figure out for yourself what the story is”

0.54

Scientific institutions are competitive and adversarial by design, which sharpens conflict and may benefit truth-seeking, but are shaped by haphazard incentives worth examining.

3 pointscentrality 4/5
  • The scientific establishment is not a monolith that meets to decide what to tell the public; it is a 'red in tooth and claw' world of competition where enormous prestige goes to whoever gets in early on something correct, so any plausible world-changing discovery—even from an outsider—would quickly be picked up by ambitious scientists seeking to make their careers.

    “It's this very red in tooth and claw world, where everybody's trying to scramble their way to the top, and there's enormous prestige and so forth to be had by getting in early on something that's right”

  • Science is more adversarial than commonly portrayed, with rival schools and camps, and this adversarial structure—analogous to a prosecutor-versus-defendant model of justice rather than consensus—may actually be good, because it sharpens conflicts in indirect data that would otherwise be ignored, whereas too tight an epistemic community can lead everyone onto the wrong track and slow science down.

    “to some degree, science is more adversarial than it's usually portrayed in the public, because you do have these different schools and these different political camps”

  • Scientific norms and institutions—publishing, the priority rule for credit, the tenure system—evolved haphazardly from Enlightenment ideas and create incentives that shape which questions scientists ask and therefore what we come to believe is true; examining and nudging these structures (as in tackling the reproducibility crisis) can make science function better epistemically.

    “those norms and institutions create the incentives that shape our behavior and the questions we ask and of course that determines the outcome we get”

0.36

Peer review and preprint systems enrich published work but do not guarantee correctness; openness and rapid critique improve science more than closed gatekeeping.

2 pointscentrality 3/5
  • Peer review functions like burn-in in engineering: it enriches the pool of published work for things that are correct and interesting by discarding much that is neither, but it does not guarantee any individual paper is correct or interesting, and the tiered journal system reflects degrees of enrichment, so a published, peer-reviewed paper still warrants skepticism.

    “I think of it as kinda like burn-in in engineering, it is enriching for things that are correct and interesting”

  • Treating disease spread as a mass-action process where individuals bump into each other at random gives a poor picture of what real disease spread looks like; modeling spread through human contact networks is necessary for an accurate understanding of epidemics.

    “if you just sort of treat a disease as this mass action situation where all of the individuals are just bumping into each other at random, you don't get a very good picture of what disease spread actually looks like”