What this covers

Nate Silver and Tyler Cowen range across forecasting, politics, and prediction in a conversation that moves between method and application. Silver stakes a position on what separates competent forecasters from pundits: not access to more data, but commitment to a formal process—establishing priors before analyzing evidence, locking in model rules in advance, and respecting what sample size constraints actually allow you to conclude. The conversation weaves between presidential elections, sports analytics, restaurant ratings, weather forecasting, and market behavior to test these ideas against real cases where forecasters succeed or fail.

The core dispute runs through prediction at the edges. Silver argues that markets and conventional wisdom perform reliably in normal conditions but exhibit large, nonlinear errors where things get strange—where sample size is small (roughly fifteen primary elections since 1972) or outcomes are rare (like an unconventional candidate winning nomination). Edge cases matter precisely because they are the consequential ones, yet they are where statistical models break down most. He traces how statistical power issues led sports analysts to wrongly dismiss the hot hand in athletics, how over-optimization toward short-term metrics distorts decision-making across domains, and how conformity bias in professional contexts suppresses unusual strategies until stakes force their adoption. On Trump's 2016 rise, Silver acknowledges his own informal probability estimate might have been sharpened by a rougher formal model, illustrating his central claim: that disciplined process beats intuition, even when both miss, because process leaves room to learn where it went wrong.

Sharpest takeaway

Silver argues that good forecasting comes less from raw data than from disciplined process: starting with priors, committing to model rules in advance, respecting small sample sizes, and recognizing that markets and conventional wisdom are usually 'pretty good' but fail badly at the weird edge cases that matter most.

  • A formal model disciplines thinking by forcing rules of evidence in advance, making the forecaster better than the average pundit
  • Sample size constraints (e.g. ~15 primary elections) cap how confident any analysis can be no matter how good the theory
  • Markets and heuristics are reliable in normal conditions but errors are non-linear and large at the edges

The argument · threads12 threads · 54 claims
0.85

Forecasting models fail on edge cases, small samples, and novel conditions due to nonlinearity and limited historical data.

10 pointscentrality 5/5
  • Many assumptions about how American politics work are based on a relatively narrow and unrepresentative slice of history—roughly post-WWII through 2000, perhaps just 1980-2000—and the Trump phenomenon reveals that nationalism, race, and racism, which are larger forces elsewhere in the world, were underweighted in that worldview.

    “a lot of assumptions a lot of people made about how American politics work are really based on a relatively narrow slice of history, post–World War II through 2000 or so, maybe even briefer, 1980 through 2000. It’s not really a lot of history.”

  • What people take to be the equilibrium baseline of stable politics and economics from the 1950s to the early 2000s may actually have been an outlier that could reverse; Trumpian constituencies have existed for a long time in parts of Europe, suggesting America may simply have gotten lucky for 50 years rather than embodying durable exceptionalism.

    “a lot of assumptions a lot of people made about how American politics work are really based on a relatively narrow slice of history, post–World War II through 2000 or so, maybe even briefer, 1980 through 2000. It’s not really a lot of history.”

  • Starting from the prior that candidates like Trump tend not to win the nomination, the historically reliable signal is not early poll performance—lots of unusual candidates poll well early and win Iowa or New Hampshire (rarely both)—but rather the ability to consolidate the field by becoming the party's consensus choice, which is the more unusual feat Trump would have to accomplish.

    “It’s the ability to consolidate the field after that, by becoming the consensus choice of the party that’s been more unusual.”

  • Silver admits that a lazy heuristic he used in thinking about Trump was to weight pro-Trump people's opinions less highly because of who they were—and he now believes this discounting of voices was lazy and possibly quite dangerous.

    “I thought the people who were pro-Trump were generally not people whose opinions I would weight as highly. And I think that’s lazy and possibly quite dangerous.”

  • When you have a small sample size—such as roughly 15 relevant primary elections going back to 1972—there is nothing you can do to make it not a sample size of 15; no matter how compelling the theoretical rationalization layered on top, the empirical evidence remains fundamentally limited.

    “when you have a sample size, let’s say it’s roughly 15, there’s nothing you can do to make it not a sample size of 15.”

  • When you have something unusual, there is a bigger error bar around it, so although an unconventional candidate like Trump may on average get less of the vote than someone polling much higher, he has a much longer tail on the upside—which is a reason not to dismiss him in the early going when everyone is a long shot.

    “when you have something which is unusual, then there’s a bigger error bar around it”

  • The classic political science finding is that voters care deeply about the economy, which makes up roughly 50 percent of what they vote about—though there are esoteric critiques that such models may be overfit.

    “people are deeply concerned about the economy, and that the economy makes up 50 percent or so of what people vote about.”

  • As the world avoids common mistakes, we are increasingly left talking only about weird, hard-to-forecast events; models work well as heuristics in normal conditions but handle edge cases poorly due to nonlinearity and small sample sizes—the analytical advantage of models lies mainly in the baseline cases, and forecasters should evaluate the territory covered in routine plays rather than only the spectacular ones at the edge of the range.

    “A lot of models are good heuristics, when conditions are fairly normal. They don’t deal all that well with the edge cases, because they’re fully designed, or because you have nonlinearity, or because they have small sample sizes”

  • Although less glamorous, Senate and House races provide larger quasi-independent samples that make a better test for forecasting models than presidential elections; the danger is correlated error—as in the most recent Senate races where polls were off by three or four points all in the same direction, causing underdog Republicans to win.

    “having larger quasi-independent samples, that would be the better test, ultimately.”

  • The traditional two-dimensional left-right political spectrum is a strange, contrived artifact of messy coalition building between parties; the rise of voice for quirkier ideologies (via social media) is arguably a good thing for democracy because those ideologies are no less intellectually coherent than the standard Democratic-Republican axis—though filter bubbles where people avoid unpleasant facts are a real and concerning phenomenon.

    “the traditional two-dimensional political spectrum is a strange and contrived thing, too. It’s the result of a very messy process of coalition building between parties.”

0.67

Rigorous statistical models beat informal judgment by forcing advance specification of rules and evidence.

4 pointscentrality 5/5
  • A forecaster is probably no better than the average pundit unless they have a model: building a statistical model commits you to rules and forces you to specify in advance things like at what point early polls become predictive, so doing your thinking ahead of time and setting up rules of evidence has a powerful disciplining effect—and his own informal low-probability call on Trump might have been improved by even a fairly bad formal model.

    “The good thing about building a statistical model is that it commits you to rules”

  • If the baseline is making half your decisions right and analytics gets you to 53-55 percent right, that is a big gain at the margin—but in a sport as noisy as baseball it takes a long time to show up, whereas less-noisy basketball is the better example of analytics working.

    “if the baseline is making half of your decisions right, and you make 55 percent of your decisions right, or 53 percent right, because you’re using analytics, that’s a pretty big gain at the margin.”

  • FiveThirtyEight's sports algorithms have tended to beat Vegas—not by a lot, winning about 52 percent of the time on average—which is actually a meaningful edge in betting terms.

    “the various sports algorithms that we have at FiveThirtyEight have tended to beat Vegas — not by a lot, but we’ll win 52 percent of the time and so forth on average.”

  • The long-held sabermetric claim that the hot hand is false and outcomes are random to a first approximation now appears mistaken: it was a classic error of using a low-power statistical test, which can cause an ambiguous result to be mistaken for a negative result; newer, less noisy data shows there is real streakiness in human performance.

    “it’s a classic mistake, but if you have a test that has low power, then you may mistake an ambiguous result for a negative result instead.”

0.59

Good judgment requires balancing skepticism of crowds with respect for market wisdom and maintaining cognitively diverse counsel.

6 pointscentrality 4/5
  • Even a skilled forecaster can beat markets only by a very small amount—not enough to make up the variance—because the more worshipful people become of markets the less useful they become, but markets remain pretty good and the hard part is knowing precisely when they're wrong.

    “Maybe by a very small amount, but not by enough to make up the variance.”

  • A key personal quality of good predictors is a certain mistrust of conventional wisdom held in tension with respect for the wisdom of crowds and markets—because people behave in herds, and the political press corps is the perfect example: not 500 really smart people but one or two smart people and 489 followers reading each other on Twitter.

    “You have to have a certain mistrust of conventional wisdom, and that’s a tricky thing.”

  • It is incredibly valuable to have a deputy who, without intervention, agrees with you about 80 percent of the time and is right as often as not on the other 20 percent; if agreement reaches 95 percent they are a sycophant (probably bad), and if it drops to 60 percent you might as well do the work yourself.

    “It’s incredibly valuable to have someone who, without your intervention, agrees with you 80 percent of the time. Then the 20 percent of the time that they don’t agree with you, that they’re right as often as not.”

  • From a free-market and libertarian stance, people should be given information about their own health whether or not they use it well, because it is their right to have it—even though it's unclear whether more data actually leads to better health decisions.

    “I guess I’m enough of a free marketer that I say you should give people the information whether they use it well or not. It’s their right to have it.”

  • FiveThirtyEight's own analysis concluded in a week or two that Uber, by and large in New York, was not adding cars to the streets—at least not in Manhattan—reaching the same conclusion New York City spent roughly $2 million to study.

    “Uber, by and large in New York, was not adding cars to the streets, at least not in Manhattan.”

  • Silver finds crowdsourcing forecasts boring as a journalist and resists publishing aggregated superforecaster projections because the joy and value lie in the process and heuristics of forecasting; publishing projections is almost beside the point when sample sizes are too small to tell you much, and over-focusing on projections takes people away from thinking about process.

    “I’m very process-driven as a person. For me, a lot of the joy is in thinking through the process of it”

0.53

Models must be explicable, stress-tested at extremes, and grounded in defensible assumptions rather than opaque machinery.

4 pointscentrality 4/5
  • Agent-based modeling—simulating underlying mechanisms, as weather forecasting builds a physical model of the atmosphere resolved mathematically rather than statistically—can yield insights regression cannot when you truly know how agents behave as a system, but if your assumptions are even somewhat wrong things can go haywire fast, which is why testing models against weird edge-case inputs matters.

    “weather forecasting is not particularly statistically-driven as we would think of it. They actually are creating a physical model of the atmosphere which they are resolving mathematically.”

  • A model can be approximately linear in normal ranges but must not break at the edge cases—a model implying Hillary Clinton gets 106 percent of the vote in DC is a warning sign—so building the correct functional form that applies when 'the going gets weird' matters precisely because the weird cases are the interesting ones.

    “Maybe you have a function that’s approximately linear, but can’t be at the edge case.”

  • The best long-run approach to communicating data is to explain to people why a model is actually not that complicated and rests on defensible assumptions—peeling away the BS—rather than arguing from authority that 'this is rigorous and objective so believe the numbers,' because the explanation is where trust and value come from.

    “that approach works a lot better in the long run than the approach of saying, arguments from authority, “Well, this is rigorous, and empirical, and objective, so therefore believe the numbers.””

  • As a journalist Silver highly prefers regression-based modeling to machine learning, because with machine learning you can't really explain anything and to him the whole value of a model is in the explanation—simple, explicable models that you can benchmark and whose limitations you understand beat opaque 'magic machine' outputs in the long run.

    “I highly prefer — this is going off on a tangent — but I prefer regression-based modeling to machine learning, where you can’t really explain anything. To me, the whole value is in the explanation.”

0.50

Data-driven optimization creates dangerous myopia by overweighting measurable short-term outcomes at expense of long-term effects.

3 pointscentrality 4/5
  • Because you can measure short-run outcomes (like website traffic 36 hours out or what makes you happy tomorrow) but not the long run, and can measure some things but not others, data-driven optimization causes over-optimization for short-term equilibria, making decisions myopic in a way that is a bigger problem than people realize.

    “There’s a lot of over-optimization. That’s a problem across almost any sector you’d want to talk about where data is being used. You’re optimizing for a short-term equilibrium, and it’s much harder to measure the long term.”

  • Despite real global good news—poverty rates dropping, income inequality going down, diseases being eradicated—media culture tends to focus its lens on negative aspects of society, which lowers people's happiness levels and creates a distorted, unrepresentative picture of the world.

    “There is a lot of wonderful news in the world in terms of poverty rates going down globally, income inequality going down, diseases being eradicated, but I wondered, to some extent, how much the media culture tends to focus a lens on negative aspects of society, lower people’s happiness level”

  • Weather forecasts exhibit a wet bias (over-forecasting storms) the further downstream you go toward local TV meteorologists, who are incentivized to get higher ratings, whereas the government's underlying data is very well calibrated and lacks a wet bias (apart from a few winter-weather models).

    “It’s true the further downstream you go, so the local meteorologist here in Virginia or in Washington on TV, they want to get higher ratings.”

0.47

Forecasters should scrutinize major outlets' non-obvious analysis and remain skeptical of authority-based claims of rigor.

2 pointscentrality 4/5
  • FiveThirtyEight's edge over outlets like the New York Times is its willingness to treat the media as a political actor rather than a benevolent umpire—recognizing that the way events are covered creates feedback loops affecting voter views, especially in momentum-driven primaries—while transparently distinguishing fact, opinion, analysis, and provocation and speaking in the first person rather than the 'voice of God.'

    “I think having that skepticism and seeing the media as a political actor instead of a benevolent umpire is to a first approximation the right way to do things”

  • A source of healthy skepticism is to assume that when major outlets like the New York Times or Post offer a non-obvious piece of analysis (not basic facts), there might be roughly a 40 percent chance it is basically wrong—a stance that drives both FiveThirtyEight's strengths and some of its failings.

    “when there’s a piece of analysis that isn’t necessarily obvious, to say, “Boy, there might be a 40 percent chance that that’s basically wrong.””

0.44

Forecasting success is temporary; winning models and forecasters are quickly replaced when conditions shift.

1 pointcentrality 4/5
  • It may be that the 'true model' of markets is always shifting and successful forecasters (Dalio on real interest rates, Lynch on consumer products, Buffett on value investing) enjoy a three-to-five-year run when their one or two good insights match conditions, after which the model shifts and they are replaced—so an unadapted run of even three to five years is on the extreme high end of fortunate.

    “if you can extend that by three or five years without adapting and evolving, that’s on the extreme high end I think. Three or five years is a very long and fortunate run.”

0.44

Applied analytics and heuristics improve decision-making across domains from gambling to restaurants to neighborhoods.

10 pointscentrality 2/5
  • Working even a small amount on vacation has a hugely disproportionate cost: based on his experiences (phoneless in Hawaii versus building an election model in Thailand), working about 20 percent of the time reduces enjoyment by roughly 70 percent.

    “A little bit of work, working 20 percent of the time, I think reduces your enjoyment by 70 percent.”

  • When a manager disagrees with what an employee is doing there are three fundamental strategies—capitulate (give up the battle to preserve morale or energy), fiat (assert authority because you sign the checks), or persuade (the ideal but very time-consuming option)—and the key skill is figuring out which to use and in what ratio.

    “Basically there are three strategies, three fundamental strategies of management when you have a disagreement with something that your, one of your employees is doing.”

  • Soccer analytics is at a very early stage because almost no granular data has historically been collected—for the long term only goals and bookings (red and yellow cards) exist, with no historical assists, tackles, or crossing passes—leaving large room for improvement, though existing heuristics correctly devalue goalkeepers.

    “the only data that’s been kept in the long term in soccer is goals and bookings, red cards and yellow cards.”

  • Gambling and fantasy sports are a good way to teach people applied analytics and critical thinking, with a measurable benefit to society; many Westernized countries (e.g. UK betting shops like Ladbrokes) normalize gambling without grave societal harm, so on a close call people should be allowed to do it.

    “gambling and fantasy sports are a good way to teach people applied analytics. I’m not being jokey, like, I think this has probably a measurable benefit to society.”

  • Silver remarks he should have taken half an hour to think about the opening question.

    “That’s a pretty heavy question. I should have taken half an hour to think about that.”

  • Daily fantasy sports takes away the social and rooting benefits of a season-long fantasy league and becomes a brute-force exercise—Silver wrote a program to randomly generate and load 200 high-scoring lineups at a time, which took all the joy out of it.

    “The thing about daily fantasy sports is that a lot of that is really taken away. It’s very much a brute force approach to watching sports.”

  • New York has three distinct culinary worlds—Michelin-starred high-end, hip Soho/Williamsburg, and ethnic New York—each requiring different ordering heuristics; in hyper-competitive high-end restaurants the 'order the weirdest thing' rule fails because the menu can't afford to mislead and you should order what it points you toward, whereas in places like Queens (where food is often better pound-for-pound) the weird-order rule still works.

    “there are kind of three New Yorks from a culinary perspective ... rich Michelin-starred New York, there’s kind of hip Soho and Williamsburg New York, and there’s ethnic New York”

  • New York is extremely efficiently priced at the neighborhood level: a FiveThirtyEight analysis for New York Magazine found cost accounts for an R-squared of about 0.93 with their quality index, meaning it is really hard to improve your lot by neighborhood choice.

    “cost accounts for an R-squared of like 0.93 with our quality index or something. It’s really hard to improve your lot in New York, at least at the neighborhood level.”

  • When evaluating a restaurant on Yelp or TripAdvisor, the number of reviews (especially relative to how long a place has been open) is a better signal than the average star rating, because ratings are biased: products gravitate toward four stars, niche items get rated by self-selected enthusiasts (e.g. 9/11 conspiracy books rated well by conspiracists), and people decline to leave deserved negative reviews for small mom-and-pop places.

    “I’d rather look, if you’re looking at Yelp or TripAdvisor, the number of reviews is a better signal than the average star rating. Especially the number of reviews relative to how long a place has been open.”

  • Giving up a little freedom through a few seemingly strange constraints—as in Singapore's tight rules or Germany's rigid shop-closing hours—can paradoxically produce more freedom and good outcomes, captured by Björk's lyric 'I thought I could organize freedom, how Scandinavian of me.'

    “If you give up a little bit of freedom to have more freedom, it’s like a Björk lyric, which is, “I thought I could organize freedom, how Scandinavian of me.” Singapore feels a little bit that way, too.”

0.43

Analytics reveals hidden truths in sports but requires large samples and low noise to detect meaningful effects.

5 pointscentrality 3/5
  • Analysis of many Senate races over time—where ideology is measurable through legislative records and sample sizes are large—shows there is a price for extremism: a candidate like Ted Cruz would probably cost about three or four points relative to the median generic Republican, a penalty that can be overcome only under conditions like a recession or a flawed opponent.

    “You can see that there’s a price for extremism. Not a price that can’t be overcome, if we go into a big recession or if Clinton or Bernie has huge problems, but Cruz would probably cost you three or four points relative to the median generic Republican.”

  • Barry Bonds drawing roughly 161 intentional walks in 2001 when the next closest player was about 50 is the biggest outlier in sports records—there is no other example where the record holder has three times the total of the nearest highest player.

    “There is just no other example I can think of in sports where the record holder has three times the sum total of the nearest highest player.”

  • Modern analytics revealed that defense in baseball is quite a bit more important than people would have thought decades ago—an ironic case where advanced data confirmed conventional scout wisdom rather than overturning it.

    “If anything, defense turns out to be quite a bit more important than people would have thought. It was an ironic way when some of the conventional scout wisdom was confirmed, as data got more advanced and more sophisticated.”

  • Newer, less-noisy data measuring how hard the ball comes off a hitter's bat allows predicting batting average or on-base average up or down by 20-30 points from baseline—a relevant difference in baseball that could justify, for example, demoting a leadoff hitter whose true current OBP is .340 rather than the assumed .370.

    “it looks like you can maybe predict batting average up or down, or on-base average 20 or 30 points from a baseline, which in baseball terms is pretty relevant.”

  • Teams become more strategically optimal when they have more on the line: in the World Series closers are used better (brought in the seventh or eighth inning), and in the NFL playoffs teams go for two and go for it on fourth down more often—revealing that when stakes are low culture prevails, but when only the outcome matters, optimal strategy wins out.

    “all of a sudden, baseball teams and football teams become, in general, more strategically correct when they have more on the line.”

0.40

Reputational risk and conformity pressures distort judgment and suppress innovation across forecasting, sports, and strategy.

4 pointscentrality 3/5
  • Becoming someone with more reputational risk—rather than a total outsider—may, to a first approximation, make someone a worse forecaster, because the incentive to protect reputation distorts judgment.

    “going from someone who was a total outsider, to someone who has more reputational risk. To a first approximation, I think it might make someone a worse forecaster, potentially.”

  • Drawing on Freakonomics, hockey players born in January have a powerful advantage over those born in November or December simply because they start slightly earlier than their peer group; if such a minor factor profoundly affects outcomes, then a far more identity-central factor like growing up gay in a non-accepting society—requiring large amounts of energy and bandwidth—plausibly creates a comparative disadvantage that suppresses the number of openly gay elite athletes.

    “there’s data from Freakonomics about how hockey players who are born in January, just because they start a little bit earlier than their peer group, that’s a very powerful effect versus being born in November or December instead.”

  • Few players learn the knuckleball despite high pay for two reasons: diminishing returns (a second knuckleballer in a league discernibly hurts both pitchers' success because batters adjust) and, more fundamentally, that sports—like many walks of life—engender powerful pressures to conform that suppress unusual approaches.

    “there are diminishing returns in the number of knuckleballers in the league, so when you have a second knuckleballer in the National League ... that already discernibly affects the success of both pitchers.”

  • If the comparative-disadvantage theory is true, then as being gay has become more normalized for kids growing up in middle and high school, that generation should produce substantially more openly gay athletes, appearing first in individualistic individual sports (tennis, golf) and the more expressive NBA culture rather than the conformist culture of baseball.

    “as it’s become more normalized, and now people who are growing up in middle school and high school where being gay is not as much of a disadvantage. Then you’d expect from that generation there to be substantially more gay athletes.”

0.34

Metrics and comparison baselines must be adjusted for context, era, or conditions to enable fair and accurate evaluation.

2 pointscentrality 3/5
  • Just as baseball stats must be adjusted for era—50 home runs mean less in the steroid era—political metrics like candidate favorability or approval should be era- or 'park-adjusted'; for example, an unfavorable matchup between two historically unpopular candidates may need adjustment, and Obama's 48 percent approval might be a 56 'park-adjusted' approval given the climate.

    “in baseball, you have to adjust stats for the era, where if you’re in the home run or steroid era, then 50 home runs doesn’t matter as much. Maybe now, Obama with a 48 percent approval rating, is at a 56 park-adjusted approval rating.”

  • According to Silver's paper, an individual voter in Virginia has roughly a 1-in-10-million chance of swaying a presidential election—the highest of any state—so on that logic Virginia is where your vote most matters.

    “It’s like 10 percent divided by 4 million or something?”

0.06

Legal practice and healthcare administration resist analytics because they prioritize precision or lack analytical culture.

2 pointscentrality 1/5
  • The legal sector may be the last field to benefit from data analysis because law relies on precision—wanting a very precise (and possibly wrong) answer—whereas statistical methods deliver approximately-right answers, which is the opposite of what legal practice values.

    “the legal sector, I think, relies more on precision. You want a very precise and possibly wrong answer, which is what you’re trying to avoid sometimes when you’re doing statistical analysis.”

  • Despite obviously rigorous work in other respects, doctors and hospital administrators are not known for being terribly analytics-driven.

    “Doctors are not known for being terribly analytics driven. I don’t know the culture enough as to know why.”