selection bias
A distortion in data caused by non-random selection of the sample, which can make observed differences (e.g., between donor types) misleading.
As defined in et_491
Where it comes up · 24
- Joshua Angrist on Econometrics and Causation3 claims
- Brian Nosek on the Reproducibility Project3 claims
- James Heckman on Facts, Evidence, and the State of Econometrics3 claims
- Andrew Gelman on Social Science, Small Samples, and the Garden of the Forking Paths2 claims
- Alberto Alesina on Immigration and Redistribution2 claims
- 345 - Resilience2 claims
- Alex Telford - Unlocking Innovation in Pharma - [Invest Like the Best, EP.360]2 claims
- Aswath Damodaran - Making Sense of the Market Pt. 2 - [Invest Like the Best, EP.349]2 claims
and 16 more
Most important claims · 10 of 36Search all claims for “selection bias” →
- P-hacking and the garden of forking paths are not fraud or deliberate manipulation; they are ordinary research practice where the natural inclination to try different specifications when an initial result is uninteresting produces spurious significance, comparable to an unblinded doctor using a biased procedure with good intentions.Andrew Gelman on Social Science, Small Samples, and the Garden of the Forking Paths
- In a small, highly variable sample the standard error is large, so any result that can reach statistical significance must be large (about two standard errors from zero); because only large estimates clear the significance bar, published significant effects are systematically overestimated due to selection bias.Andrew Gelman on Social Science, Small Samples, and the Garden of the Forking Paths
- Public judgment of Knox's behavior rested on selection bias: the media replayed isolated moments like the kiss with her boyfriend in slow motion while ignoring context—that she had known the victim only three weeks, never saw the body, and broke down crying in private—so people projected guilt onto curated fragments.345 - Resilience
- The media created a selection bias in how Knox's behavior was interpreted: media outlets zoomed in on moments when she was not visibly crying or showed 'inappropriate' emotional responses, while ignoring the context that she had not witnessed the crime scene directly and did not yet understand the full horror of what had occurred.Resilience: A Conversation with Amanda Knox (Episode #345)
- Many apparent cases of 'what doesn't kill you makes you stronger' are illusions of selection bias. The stressor kills or weeds out the weak ones rather than strengthening survivors. This is effectively how stress works within cells, more destructive than strengthening, and it applies to populations too.Nassim Taleb on Antifragility 01/16/2012
- Fryer's findings are limited because the data come only from Houston—a department willing to share data perhaps precisely because it is exculpatory and unrepresentative—and because relying on arrest data requires assuming the arrest process is racially unbiased; if police arrest relatively less-threatening black people more readily, then a comparable shooting rate actually proves discrimination rather than its absence.42 - Racism And Violence In America
- To avoid selection bias when a paper contained multiple studies, the project by default selected the last study reported in the paper and its key finding, only moving to an earlier study if the last one was infeasible to replicate, thereby preventing teams from choosing studies they thought most or least likely to replicate.Brian Nosek on the Reproducibility Project
- Book blurbs carry no real information about quality because they only ever have positive valence--no one writes a negative blurb--so all a blurb actually conveys is whether you recognize the identity of the person who wrote it, which shifts what counts as a worthwhile book from its contents to how blurbed it is and who endorsed it.The Invisible Hierarchies that Rule Our World (with Toby Stuart)
- Naively comparing employment across high- and low-minimum-wage places is misleading because places that raise the minimum wage are systematically different (e.g., booming tech cities with high rents and price-insensitive customers), so a policy that works in San Francisco or Seattle may fail in rural Iowa.Susan Athey on Machine Learning, Big Data, and Causation
- The Reproducibility Project drew from 488 possible articles published in 2008 in three psychology journals (Psychological Science, JPSP, and JEPLMC), of which about 165 were eligible, 113 replications were started, and 100 were completed, with study selection constrained by feasibility of the replication team's resources.Brian Nosek on the Reproducibility Project