BuildAlpha

First, I need to describe over-fitting or more commonly known as curve-fitting. Curve-fitting is creating a model that too “perfectly” fits your sample data and will not generalize well on new unseen data. In trading, this can be thought of as your model too closely fits the historical data and will surely fail/struggle to adapt to new live data. Here are two visuals I found to help illustrate this idea of curve-fitting. So how can we avoid curve-fitting? The simplest and best way to avoid curve-fitting is to use “Out of Sample” data. We simply designate a portion of our historical data (say the last 30% of the test period) to act as our unseen or “Out of Sample” data. We then go about our normal process designing/testing rules for trading or investing using only the first 70% of the test period or the “In Sample” data. After finding a satisfactory trading method we pull out our Out of Sample data and test on the last 30% of our test period. It is often said ...