causal

Multiple Comparisons Invalidate Significance Tests

Running many low-powered studies or trying many specifications and then reporting only the final result that passes p < .05 makes the standard significance criterion irrelevant, because the cumulative chances of finding an effect are not accounted for in the single reported test.

causalpending

Speaker

Brian Nosek

Evidence Quote

“still using the same criterion: p less than .05 just on the one final one you report”

Source

Brian Nosek on Truth, Science, and Academic Incentives— EconTalk
Created: 6/13/2026, 12:17:26 AM

Your notes

Loading notes...