Sheela
Sheela
Home
Add
Questions
Thinkers
Claims
People
Concepts
Jargon
Sources
Health
Predictions
Collections
Domain Landscape
Contribute
Review
Dedup
Topic requests
Settings
Help

← Jargon

Jargoneconomics· 2 claims· 1 episode

efficacy test

A test conducted with the best possible inputs to see whether an idea can work under ideal conditions, as opposed to whether it works at scale.

As defined in John List on Scale, Uber, and the Voltage Effect


Where it comes up · 1
  • John List on Scale, Uber, and the Voltage Effect2 claims

Most important claims · 2 of 2Search all claims for “efficacy test” →
  • A/B tests and efficacy tests use the best inputs a researcher can marshal, but those best inputs cannot be replicated at scale; failing to disclose that a result was an efficacy test causes voltage drops when policymakers later try to scale it.John List on Scale, Uber, and the Voltage Effect
  • There are five reasons (vital signs) ideas fail to scale: (1) false positives—the original result was an unrepresentative bad sample; (2) failing to know your audience—overestimating how many people the program can help; (3) failing to know your situation—mistaking an efficacy test for a scalable one; (4) spillovers—market-wide equilibrium effects and behavioral spillovers like moral hazard; and (5) supply-side returns to scale—whether the idea has economies, constant, or decreasing returns to scale.John List on Scale, Uber, and the Voltage Effect