Controlled experiment
A test in which a change is shown to one group of users while a comparable group is unchanged, allowing the change's causal effect to be isolated and measured.
As defined in Adam D'Angelo on Knowledge, Experimentation, and Quora
Other definitions
An experiment that isolates one factor by holding other conditions constant to test cause and effect.
364 | Stuart Firestein on How Science Relies on Ignorance and Failure
Where it comes up · 2
- Adam D'Angelo on Knowledge, Experimentation, and Quora3 claims
- 364 | Stuart Firestein on How Science Relies on Ignorance and Failure1 claim
Most important claims · 4 of 4Search all claims for “Controlled experiment” →
- Personal usage is good for generating theories and ideas, but those ideas should ideally be tested with controlled experiments and cross-checked against surveys and other data sources, because acting on untested intuition can send a product in a crazy, destructive direction.Adam D'Angelo on Knowledge, Experimentation, and Quora
- Quora built an experiment framework about four years ago, earlier than most companies do, and since then has run about 2000 controlled experiments, with roughly 30 running concurrently at any time, staffed by 13 data scientists out of a 130-person company.Adam D'Angelo on Knowledge, Experimentation, and Quora
- Most product experiments fail to confirm their hypotheses, and Quora's success rate is around a third, which is actually better than rates D'Angelo hears from other companies; this is counterintuitive and demoralizing to newcomers because nearly everything they did before usually appeared to work.Adam D'Angelo on Knowledge, Experimentation, and Quora
- Bacon's invention of the experiment meant isolating a slice of nature, holding other factors constant, and probing it actively, which is distinct from trial-and-error experience.364 | Stuart Firestein on How Science Relies on Ignorance and Failure