Sheela
Sheela
Home
Add
Questions
Thinkers
Claims
People
Concepts
Jargon
Sources
Health
Predictions
Collections
Domain Landscape
Contribute
Review
Dedup
Topic requests
Settings
Help

← Jargon

Jargonmachine learning· 4 claims· 1 episode

test set

Data the model never saw during training or tuning, used to get an unbiased estimate of performance on new examples.

As defined in Machine Learning in Investing


Where it comes up · 1
  • Machine Learning in Investing4 claims

Most important claims · 4 of 4Search all claims for “test set” →
  • The test set must be used only once the model is finished, because if it is used to guide tweaks it becomes part of the training feedback loop and can no longer provide an honest out-of-sample estimate.Machine Learning in Investing
  • Time-series data should be split by time, training through some year and testing forward, because splitting across unrelated sectors or random dates lets the model learn from future correlations, which corrupts the test.Machine Learning in Investing
  • The test set must consist of examples the model has never seen during training, so the model cannot cache answers and the evaluation gives an honest estimate of performance.Machine Learning in Investing
  • Models are trained on a training set, tuned against a validation set, and only then assessed on a true out-of-sample test set that was never used for any decision, which is what makes the final performance estimate credible.Machine Learning in Investing