efficacy test
A test conducted with the best possible inputs to see whether an idea can work under ideal conditions, as opposed to whether it works at scale.
As defined in John List on Scale, Uber, and the Voltage Effect
Where it comes up · 1
Most important claims · 2 of 2Search all claims for “efficacy test” →
- A/B tests and efficacy tests use the best inputs a researcher can marshal, but those best inputs cannot be replicated at scale; failing to disclose that a result was an efficacy test causes voltage drops when policymakers later try to scale it.John List on Scale, Uber, and the Voltage Effect
- There are five reasons (vital signs) ideas fail to scale: (1) false positives—the original result was an unrepresentative bad sample; (2) failing to know your audience—overestimating how many people the program can help; (3) failing to know your situation—mistaking an efficacy test for a scalable one; (4) spillovers—market-wide equilibrium effects and behavioral spillovers like moral hazard; and (5) supply-side returns to scale—whether the idea has economies, constant, or decreasing returns to scale.John List on Scale, Uber, and the Voltage Effect