John List on Scale, Uber, and the Voltage Effect
What this covers
John List, an economist at the University of Chicago who served as chief economist at both Uber and Lyft, discusses with Russ Roberts what separates ideas that genuinely work from those that merely look good in controlled settings. List argues that the only meaningful measure of an idea's worth is whether it scales, and that most promising programs encounter what he calls the Voltage Effect—a collapse in efficacy when moving from small-scale testing to real-world deployment at scale. The conversation draws on List's time inside ride-sharing platforms to illustrate how constraints that don't appear in an efficacy test become decisive once an idea reaches full size.
List identifies Five Vital Signs of Scaling: false positives in the original sample, misreading your audience, confusing testing conditions with scalable ones, spillover effects from market-wide behavior, and supply-side economics that may not sustain gains. His Uber tipping experiment becomes a central case study—when the platform added tipping nationwide, new drivers entered the market and existing drivers worked longer hours, generating enough empty-car driving to completely offset the wage gains riders were supposed to provide. The conversation explores why attempts to raise driver pay through various means all fail in the same way: free entry competes away rents as long as drivers can seek opportunities whenever they wish. List contrasts this with surge pricing, which clears markets through price rather than wait times, and explains why predictable versus unanticipated surge produce vastly different driver responses. The discussion also examines thin-slice data and marginal versus average thinking—why the most recent 25–50 customer acquisitions predict future outcomes better than averages across thousands—and the distinction between horizontal and vertical scaling, showing why scaling a program across many cities differs radically from scaling it within one city where labor supply tightens.
List argues that whether an idea is genuinely good depends on whether it scales, and that most promising programs suffer a 'voltage drop' because efficacy tests with ideal inputs fail to anticipate the supply, audience, situation, and spillover constraints that emerge at scale.
- Five 'vital signs' (false positives, audience, situation, spillovers, supply-side returns) determine whether an idea retains its effect at scale
- Uber's tipping experiment showed an Econ-101 equilibrium where added supply offset wage gains, illustrating voltage drop in practice
- Marginal (thin-slice) thinking beats average-based decisions when conditions change
Regulations claiming one justification often serve rent-seeking interests of incumbent asset holders.
- While New York's rideshare supply restrictions were justified by congestion, they likely also reflect medallion holders' rent-seeking to maintain the value of their assets, fitting the bootlegger-and-Baptist pattern of regulation.
“I suspect that the restrictions on supply that New York City has imposed--true, they could argue it was for congestion--but I'm sure some of it was a response to medallion holders trying to maintain the value of their asset.”
Scalability requires replacing dependence on exceptional talent with standardized procedures average practitioners can reliably execute.
- McDonald's is successful at scale because it developed procedures that a merely competent cook can execute, whereas a business requiring a great chef cannot scale into a chain of tens of thousands of locations.
“the reason McDonald's is successful is that they figured out a set of procedures that someone could execute who was pretty good at cooking, but not a great chef. And that's--if you need a great chef to run your restaurant, by definition you can't have a big chain.”
Measurement must isolate the specific causal factor being tested, not conflate it with other variables.
- Testing a curriculum with the 30 best teachers does not measure the marginal impact of the curriculum, because it conflates the curriculum effect with the effect of exceptional teachers; using standard-quality teachers isolates the curriculum's contribution.
“if it was going to be a real test of the curriculum, having the 30 best teachers teach your curriculum is not measuring the marginal impact of the curriculum. It's a combination of curriculum with the best teachers.”
Trust mechanisms shift from brand-based screening to technology-enabled tracking and peer rating systems.
- Whereas taxi passengers trust the brand to screen drivers, Uber riders get into a stranger's car because they trust the GPS tracking and the rating system to do the filtering and monitoring, an extraordinary and underappreciated social shift that also underlies Airbnb but depends on people giving honest ratings.
“we step into the car of a taxi driver because we trust the brand... I get into the Uber... and the reason I get into them is I trust that GPS system and I trust the rating system to do the filtering.”