causal
A/B Tests Are Biased Toward Short-Term Gains
Tech firms systematically innovate in the wrong dimensions because running long experiments is expensive, biasing them toward short-term experiments; changes that confuse users initially but help long-term get penalized, so summing the short-term gains across years might predict 300% revenue improvement while actual improvement was only 10%—they excel at incremental short-term innovation but miss more radical changes.
causalpending
Speaker
Susan AtheyEvidence Quote
“they tend to systematically innovate in the wrong dimensions. So they are awesome at incremental short-term innovation... But if there is something more radical, they'll sometimes miss it.”
Created: 6/13/2026, 12:14:45 AM
Your notes
Loading notes...