|
Throughout the history of technical analysis traders have lent a considerable amount of their focus to the optimization of trading models. That is, individuals have analyzed the past, located something that appeared to have a significant trading advantage, then optimized rules and procedures to trade that specific strategy in the future. Sometimes this has worked, however, most often it has not.
In the past, traders attempted to address this problem by applying more system optimization and curve fitting. However, additional optimization and curve fitting is not the answer considering this is the root of the problem. It would be the same concept as trying to extinguish a fire with additional fire.
If given enough optimization and curve fitting, an individual can apply just about any trading strategy or procedure to a given financial entity and force it to appear profitable. However, utilizing this approach, will usually lead to a trading model that trades the past well but will most likely fail significantly in the future.
I could easily write an entire book dedicated to the subject of proper trading model design and its mysterious sounding terms such as swarm simulation, Monte Carlo analysis, and genetic algorithms. Not to take away from these advanced techniques, as there certainly have there crucial place within my trading system analysis, but they are very simple steps one can take to avoid building over optimized and curve fitted trading models that typically only exploit the past well but fail in the future.
By utilizing the out-of-sample walk forward approach and testing those system parameters over various markets, one can simulate a more realistic trading scenario. If a system can hold up on a walk forward test and demonstrate profitability over various markets we most likely have extrapolated events that can lend themselves to profitable trading in the future.
So how do you apply simple out-of-sample testing using the walk forward approach to trading model construction? By initially building your trading rules utilizing only a segment of an entire data series, then back testing a strategy, and lastly testing a system on future data that has been eliminated from consideration while designing and back testing the model.
For example, assume we have a potential strategy for trading the S&P 500 futures contract. After programming a system’s parameters into a trading platform we must choose our data period for testing. Let say we have 15-years of back-adjusted continuous S&P 500 futures data on file from January 01, 1985 through January 01, 2000. To utilize the out-of-same walk forward approach we will build our strategy with only accessing the data period from January 01, 1992 through December 31, 1994. Next we backtest our strategy from January 01, 1985 through December 31, 1991. Lastly, we walk the system forward by testing the system on the data we have eliminate so far which is from January 01, 1995 through December 31, 2000.
If after a walk forward test, your strategy holds up then you most likely have something that will remain profitable into the future. Additionally, you should test you strategy across various markets. Studies have shown that a randomly chosen, curve fitted system can make money in one or two walk-forward test, but will not make money over a large number of walk-forward tests. So, you want at least 500 observations minimum within your test to reduce the probability of isolating any random occurrences.
With the correct amount of observations the out-of-sample walk forward approach represents a real world trading simulation of the way a trading system is used in real time. It also answers the following important questions:
- Will a trading model continue to make money after optimization?
- Is a trading system extrapolating random observations or something that has repetitive merit?
- What impact will changes in trend and volatility levy on future performance?
Answering the above questions provides the following benefits:
- It verifies the forward-trading ability of the model, i.e. how versatile is the model to changes in trend and volatility.
- It detects if a system has been curve fitted or improperly optimized. A system that demonstrates profitably over a large number of walk-forward tests is most likely to be successful within future applications because it generates greater statistical validity.
|
