statistically significant
A result unlikely to be due to chance by a conventional statistical threshold; achieving it is often a publication requirement, incentivizing certain research choices.
As defined in Adam Mastroianni on Peer Review and the Academic Kitchen
Where it comes up · 6
- Austin Frakt on Medicaid and the Oregon Medicaid Study2 claims
- What Modern Medicine Gets Wrong (with Marty Makary)2 claims
- Scott Atlas on American Health Care1 claim
- James Heckman on Inequality and Economic Mobility1 claim
- Adam Mastroianni on Peer Review and the Academic Kitchen1 claim
- If Life Is Random, Is It Meaningless? (with Brian Klaas)1 claim
Most important claims · 8 of 8Search all claims for “statistically significant” →
- In the 'Universe of Uncertainty' study, 76 non-communicating research teams given identical data on whether immigration affects support for the social safety net produced scattered results—about half found no effect, a quarter found a statistically significant positive effect, and a quarter found a statistically significant negative effect—revealing that even with controlled data and responsible methodologies, social research yields scattershot conclusions that publication bias then cements into 'settled' policy-shaping wisdom.If Life Is Random, Is It Meaningless? (with Brian Klaas)
- The Oregon study was statistically underpowered to detect realistic Medicaid effects on physical health: the effect sizes it could have detected were enormous (e.g., it would have needed an 80% reduction in elevated blood sugar versus the observed 20% point estimate), so the non-significant results are genuinely uninformative rather than evidence of zero effect.Austin Frakt on Medicaid and the Oregon Medicaid Study
- Twenty-two years ago an NIH doctor held a press conference announcing hormone replacement therapy causes breast cancer, scaring tens of millions of women into stopping it, but the published data from this nearly billion-dollar study (the largest taxpayer-funded study in US history) showed no statistically significant increase in breast cancer versus placebo, meaning the data were misrepresented.What Modern Medicine Gets Wrong (with Marty Makary)
- In the WHO 2000 World Health Report ranking, the differences between countries ranked number 3 and number 30 were not statistically significant -- meaning the difference could be explained by random variation -- yet they were still reported as ordered rankings, which misleads the public.Scott Atlas on American Health Care
- The lead HRT researcher had written in a journal before the study results were clear that 'we have to stop the hormone replacement bandwagon,' and before the press conference assembled 40 researchers in a Chicago hotel where they protested that the paper—already submitted with their names—did not show a statistically higher rate of breast cancer, indicating the conclusion was decided in advance of the data.What Modern Medicine Gets Wrong (with Marty Makary)
- The latest Oregon study found roughly a dozen physical health measures—blood pressure, cholesterol, blood sugar control, and related medications—none of which were statistically significant, creating the appearance that Medicaid has no effect on physical health, which is the core of the controversy.Austin Frakt on Medicaid and the Oregon Medicaid Study
- The modern incentive of publish-or-perish, combined with too many PhDs and risk-aversion, pushes scholars to work on tiny, safe questions that reliably reach statistical significance but are unimportant to the world—a worse tragedy than the wasted reviewing hours—whereas the pre-journal era's smaller pool of bolder risk-takers with low material returns produced different, riskier work.Adam Mastroianni on Peer Review and the Academic Kitchen
- Magne Mogstad and coauthors examined the small zipcode-level neighborhoods in Chetty's Atlas of Opportunity and asked whether mobility patterns in one neighborhood are statistically significantly different from another; they found the differences were not statistically significant and could arise by chance, meaning the atlas is essentially numerology rather than hard evidence that neighborhoods are predictive.James Heckman on Inequality and Economic Mobility