effect size
A quantitative measure of the magnitude of a drug's benefit observed in a trial.
As defined in Jacob Stegenga on Medical Nihilism
Other definitions
Magnitude of an observed relationship or difference between groups; in correlation research, measured by correlation coefficient r; empirical distribution of effect sizes in psychology shows most published effects are r < 0.50
2017 Personality 18: Biology & Traits: Openness/Intelligence/Creativity IMeasure of the magnitude of a treatment or intervention's impact; used in scientific analysis to determine whether changes are meaningful or just statistically significant
“I feel like I’m 25 again” - Dana White’s $120,000 Health Protocol
Where it comes up · 10
- Brian Nosek on the Reproducibility Project4 claims
- John Ioannidis on Statistical Significance, Economics, and Replication4 claims
- Finding Undiscovered Talent1 claim
- 2017 Personality 18: Biology & Traits: Openness/Intelligence/Creativity I1 claim
- Jacob Stegenga on Medical Nihilism1 claim
- Fredrik deBoer on the Cult of Smart1 claim
- 127 - Freedom From The Known1 claim
- 412 - Better Things Better People1 claim
and 2 more
Most important claims · 10 of 16Search all claims for “effect size” →
- Across the 100 replications, replication effects were on average half the magnitude of the original effects: 97% of original studies reported statistically significant results but only 36% of replications did, and only 39% of effects were subjectively rated as having replicated.Brian Nosek on the Reproducibility Project
- In statistics 'significant' means a result is unlikely to have occurred by chance, which is distinct from the everyday meaning of importance; a statistically significant effect can have an effect size so small that it is practically unimportant, as with the bacon-cancer link.Brian Nosek on the Reproducibility Project
- Operating in an underpowered environment creates two problems simultaneously: a very high chance of false negatives (missing real signals), and a high risk that any statistically significant result is a false positive or a grossly inflated effect size—the smaller the power, the greater the average inflation of detected effects.John Ioannidis on Statistical Significance, Economics, and Replication
- In an underpowered environment with very small samples, a true signal can only be detected if it happens to present itself at a much bigger magnitude than reality—because if it appeared at its true size it wouldn't be significant—so any detected result is either a false positive or an exaggerated effect, a phenomenon called the Winner's Curse.John Ioannidis on Statistical Significance, Economics, and Replication
- As trials have controlled for more biases over the last generation, they have measured a decrease in drug effectiveness—there is an inverse correlation between trial quality and measured effect size—so extrapolating, better future trials will reveal smaller benefits while better harm detection will reveal larger harms, yielding a general argument for medical nihilism.Jacob Stegenga on Medical Nihilism
- Replication success was better predicted by the strength of the original evidence than by characteristics of the original or replication teams, indicating that the robustness of the underlying finding matters more than who conducted the study.Brian Nosek on the Reproducibility Project
- Because there is no single unambiguous way to define a successful replication, the project used five different criteria—including whether the replication reached statistical significance (p<.05), whether the original effect size fell in the replication's 95% confidence interval, effect-size comparison, subjective ratings, and combined significance—so the same result can count as a success on some criteria and a failure on others.Brian Nosek on the Reproducibility Project
- In the economics topics analyzed, it is plausible that the large majority of reported effect sizes are substantially exaggerated, with a 2-fold exaggeration very common and about a third of cases inflated by 4-fold.John Ioannidis on Statistical Significance, Economics, and Replication
- We never needed a randomized trial to prove smoking causes cancer because smoking increases cancer risk roughly 20-fold, whereas many nutrients show effects of only about 1.001-fold; large effect sizes make observational designs reliable while tiny ones do not.John Ioannidis on Statistical Significance, Economics, and Replication
- A rigorous OpenAI-funded basic income study by leading economists in a rich country produced underwhelming results: positive social effects were limited or non-existent and there was a significant drop in hours worked—though that reduced work could be reframed as a feature rather than a bug if it frees people from bullshit jobs to do meaningful things.412 - Better Things Better People