factual

Data dredging invalidates classical statistical tests

When researchers iterate between data and hypotheses—trying different formulations, excluding sample points, adding or removing variables—they leave the framework of classical statistical testing in which significance tests are valid, yet they continue to report and interpret those tests as if they still applied.

factualpending

Speaker

Russ Roberts

Evidence Quote

“If, however, you start going back and forth between the data and hypotheses, trying different formulations, excluding certainly sample points, adding variables, taking some out, you've really left... the field of classical statistical testing, and the tests don't apply any more. But we pretend that they do.”

Source

Brian Nosek on the Reproducibility Project— EconTalk
Created: 6/13/2026, 12:25:28 AM

Your notes

Loading notes...