natural experiments
Real-world situations where circumstances approximate a controlled experiment, allowing researchers to identify causal effects without deliberate manipulation.
As defined in Paul Krugman
Where it comes up · 4
- Steven Levitt on Freakonomics and the State of Economics3 claims
- Paul Krugman1 claim
- Jim Manzi on Knowledge, Policy, and Uncontrolled1 claim
- Robert Shiller on Narrative Economics1 claim
Most important claims · 6 of 6Search all claims for “natural experiments” →
- When natural-experiment and causal-inference techniques were new 30 years ago, doing them was a scarce, hard talent that carried academic rewards; now there is a cookie-cutter recipe for extracting causal results from a law change, so the skill is no longer scarce and academics have moved on to complicated structural models, making economics less policy-relevant.Steven Levitt on Freakonomics and the State of Economics
- Replication in economics is narrowly defined as whether posted code reproduces the published tables, but starting from raw data most—if not all—papers contain enormous numbers of mistakes; most such errors push results toward zero rather than away from it, and full replication is unlikely to happen for most findings.Steven Levitt on Freakonomics and the State of Economics
- Krugman advises today's budding research economists not to do what he did—heavily theory-oriented simple models—but to exploit the golden age of empirical work: find data not heavily worked over, look for anomalies or natural experiments that shed light on disputed issues, and step beyond traditional economic methodology, since methods like ethnography could yield insights the standard paradigm misses.Paul Krugman
- Unlike psychology where suspected fraud carries huge penalties, economics tolerates suspected fabrication because the field's problem space is so vast relative to the number of economists that only a handful care about any one result; Levitt and List failed to replicate two outrageous experimental papers and suspected fraud, but editors forced them to remove fraud language and the author paid no price.Steven Levitt on Freakonomics and the State of Economics
- While macroeconomic experiments are difficult and ethically problematic, many controlled experiments—routine in marketing, journalism, and communications—demonstrate that people respond to stories, providing causal evidence that narratives affect human behavior.Robert Shiller on Narrative Economics
- Money supply is one of the few macroeconomic areas where economists across ideologies gained genuine knowledge—Friedman and Schwartz's Monetary History changed minds about money's effect on inflation—but this came from natural experiments where little else changed, not from complicated multivariate regressions, and even here only qualitative confidence (money creation raises the price level) is warranted, not quantification of effects on unemployment.Jim Manzi on Knowledge, Policy, and Uncontrolled