causal

Out-of-Sample Testing Is No Panacea

The popular out-of-sample method of holding out recent data is not a reliable fix because researchers already know what happened in the holdout period and can rig variables, can iteratively re-fit until something works, and a genuinely true strategy may still fail out-of-sample by bad luck over a short window.

causalpending

Speaker

Campbell Harvey

Evidence Quote

“it is not a panacea to actually go to the other sample method”

Source

Campbell Harvey on Randomness, Skill, and Investment Strategies— EconTalk
Created: 6/13/2026, 12:29:08 AM

Your notes

Loading notes...