factual

Significance in Small Samples Should Raise Suspicion Not Confidence

The intuition that finding statistical significance in a small sample implies a larger sample would yield even stronger evidence is a fallacy ('that which does not kill my statistical significance makes it stronger'); small noisy samples that nonetheless cross the threshold actually signal inflated, unreliable estimates.

factualpending

Speaker

Andrew Gelman

Evidence Quote

“You're using what we call the 'That which does not kill my statistical significance makes it stronger' fallacy.”

Source

Andrew Gelman on Social Science, Small Samples, and the Garden of the Forking Paths— EconTalk
Created: 6/13/2026, 12:23:45 AM

Your notes

Loading notes...