normative
Generating and testing hypotheses on same data is invalid
You cannot confidently generate and test a hypothesis using the same data: digging into data to find moderating influences after seeing the result means the analysis is shaped by the data, so any pattern found must be confirmed with new data rather than treated as a valid test.
normativepending
Speaker
Brian NosekEvidence Quote
“we've now looked at the data, and the data itself has shades, how we analyze that data. So we are both simultaneously generating and testing the hypothesis, with the same data. And that's a no-no.”
Created: 6/13/2026, 12:25:28 AM
Your notes
Loading notes...