causal
Multiple Testing Requires Higher Significance Bar
When the number of tests increases, the standard two-sigma (95% confidence) rule becomes unreliable because random chance will produce apparent significant findings; with 20 tests you expect roughly one false positive at the 5% level, so the cutoff must be raised as in physics' five-sigma standard.
causalpending
Speaker
Campbell HarveyEvidence Quote
“when you do 20 tests, you can't use the 2-sigma rule. So by the 2-sigma rule, if you try 20 things, then the odds are that something is going to come up as a fluke”
Created: 6/13/2026, 12:29:08 AM
Your notes
Loading notes...