pre-registration
The practice of publicly recording a trial's design and planned outcomes before it is run, intended to prevent outcome-switching and selective reporting.
As defined in Jacob Stegenga on Medical Nihilism
Where it comes up · 9
- John Ioannidis on Statistical Significance, Economics, and Replication3 claims
- Is Most Published Research Wrong?2 claims
- Andrew Gelman on Social Science, Small Samples, and the Garden of the Forking Paths2 claims
- Jacob Stegenga on Medical Nihilism2 claims
- Brian Nosek on Truth, Science, and Academic Incentives1 claim
- Vinay Prasad on Cancer Drugs, Medical Ethics, and Malignant1 claim
- Katy Milkman on How to Change1 claim
- 411 - The Victimhood Pandemic1 claim
and 1 more
Most important claims · 10 of 14Search all claims for “pre-registration” →
- Randomized trials and meta-analyses are not infallible truth-tellers; they can be shot through with biases via malleability practices analogous to p-hacking—such as measuring many outcomes and publishing only a subset (one German study found only about 26% of pre-registered outcomes were published) and publication bias of entire trials.Jacob Stegenga on Medical Nihilism
- When replications fail, the key difference is usually not the experimental conditions but that the replication is controlled—the analysis is specified in advance—whereas the original study was uncontrolled and permitted p-hacking; even honest researchers like Nosek discovered they had p-hacked without realizing it.Andrew Gelman on Social Science, Small Samples, and the Garden of the Forking Paths
- For strong confirmatory tests, writing up and registering the full hypothesis and analysis plan before accessing the data dramatically reduces researcher degrees of freedom; this is appropriate only for confirmatory work, not for genuinely exploratory research where no strong prediction exists.Brian Nosek on Truth, Science, and Academic Incentives
- Studies comparing trial publications in top journals (Lancet, JAMA, NEJM, BMJ) to their pre-registration plans found massive disparities, including switching the stated primary outcome in about half of trials, and journals' pledges to publish only pre-registered trials were not enforced.Jacob Stegenga on Medical Nihilism
- Methods are improving through pre-registration (publicly posting hypotheses and tests in advance) and through 'mega-studies' that test dozens of hypotheses simultaneously on the same outcome and timeframe, which accelerates science and makes it easier to publish null results alongside positive ones, though they carry risks of outlier skew and misrepresentation.Katy Milkman on How to Change
- Research obtained through a garden of many analytic paths and countless analyses should be honestly acknowledged as entirely exploratory data-dredging rather than dressed up as pre-registered; that is fine provided everyone knows what it is, and the specific emergent recipe can then be pre-registered and tested on a different dataset to see if it replicates.John Ioannidis on Statistical Significance, Economics, and Replication
- No single solution fixes research bias; over a dozen complementary families of solutions (replication culture, pre-registration, data sharing, better statistical methods, stronger thresholds, open peer review, transparency) reinforce each other, but they only take hold when multiple stakeholders—journals, institutions, funders, scientists—simultaneously adopt them, as clinical-trial registration only happened once major journals refused to publish unregistered trials.John Ioannidis on Statistical Significance, Economics, and Replication
- Instead of only pushing research toward the idealized assumptions that significance testing requires (pre-registration, correct identification), there is a second, often-neglected direction: developing statistical methods that more openly integrate theory with data and better reflect how we can actually learn.Andrew Gelman on Social Science, Small Samples, and the Garden of the Forking Paths
- As a consequence of psychology's replication crisis, open-science efforts (sharing datasets and code, pre-registration) have become increasingly important to prevent bad research practices like p-hacking and the file-drawer problem, and psychology has arguably emerged stronger as a result.AMA | May 2023
- Pre-registration of hypotheses and methods with peer review before conducting experiments, with guarantee of publication regardless of results so long as procedures are followed, eliminates publication bias, promotes higher powered studies, and lessens incentives for p-hacking.Is Most Published Research Wrong?