out-of-sample
Testing a strategy or relationship on data not used to discover it, to check whether it holds beyond the original sample.
As defined in Cliff Asness
Where it comes up · 2
Most important claims · 5 of 5Search all claims for “out-of-sample” →
- Every empirical regularity that predictably works (or fails) does so for one of three reasons: complete accident (data mining), a behavioral story (someone making a systematic error you exploit), or risk (in a rational world you get paid for bearing risk); these are not mutually exclusive and can coexist.Cliff Asness
- A strategy should be validated by splitting the time series into halves, fitting on the first half and testing on the second, and checking whether the performance and p-values agree; if they do, the model is unlikely to be overfit and more likely to persist.Quant vs. Traditional Investors and How Alphas Become Betas
- For crowdsourced datasets whose contributor panel changes over time, first-half versus second-half validation is preferred because it reveals whether the decision quality behind the signal is intrinsic or whether participants' heuristics and platform changes have altered it.Quant vs. Traditional Investors and How Alphas Become Betas
- For discretionary managers the behavioral edge is currently the most important one, because their decision flow is where value is won or lost, while quants also show irrationality by delaying deployment of strategies after recent underperformance.Quant vs. Traditional Investors and How Alphas Become Betas
- Quant teams are not immune to irrationality: they test strategies, see strong p-values, then hesitate to deploy them because of a recent weak stretch in a ten-year track record, and similarly turn strategies on and off emotionally.Quant vs. Traditional Investors and How Alphas Become Betas