Traditional optimization of trading strategies
Optimization is the process to adapt the parameters of a given strategy to a specific market. It is generally a good thing when done properly, but if not done correctly, there is a high risk to do it wrong and end up with a curve fitted system (I will go deeper on the traditional optimization of trading strategies on future posts).
What is curve fitting?
You can always find a combination of rules and trading parameters that fits perfectly to the available historical data, resulting in exceptional trading results based on those tests.
But when those rules are tested on a live market, they fail and loose money very quickly.
The majority of the strategies commercially available suffer from this problem. Why? Because vendors sell the strategy based on pretty tests instead of robustness of the strategy. It is much easier to sell based on a nice equity curve than to sell based on the complex optimization process to increase robustness and reduce curve fitting. Sad but true.
The proper way to optimize
To avoid curve fitting, one must leave at least 30% of the available data out of the optimization process. For example, if you have data from 2000 to 2012 (12 years), the optimization process would be:
- Optimize for 2000 - 2009: You will end up with the best parameters on this period.
- Select the best parameter set: The criteria to choose it is important. For example, selecting the best AbsoluteProfit/RelativeDrawdown that has robust enough neighbor parameters is a good choice.
- Test the parameter set on the out of sample period (2010-2012): If you get different results on this period there is something wrong on previous phases. You must return to the design phase.
The problems of traditional optimization
Traditional optimization is good, but there are obvious problems:
- The chosen parameter set is average quality: As it needs to survive a lot of market conditions (both on in-sample and out-of-sample data) it will not be really adapted to any one, so many trading opportunities will be missed.
- Degradation forces periodic reoptimizations: As the time passes, the chosen parameter set degrades. We will have more data and we will have more different market conditions.
- Most of the gains could be focused on small periods: You will find many strategies out there cuve fitted to recent market conditions but when testing them on previous years they fail.
- A dramatic and definitive change in a market will make the strategy reach the worst case scenario, loosing a lot of money
There are ways to solve most of the above problems in a traditional way. But using Walk Forward hopefully we should be able to overcome or reduce the impact of all of them.
Walk Forward Analysis
With Walk Forward Analysis, instead of making one big optimization on in-sample data and testing it on the out-of-sample data, we will make a lot of small optimizations and testings on much smaller periods.
The size of the optimization period is called the "Optimization Window". According to my tests, the size of the Optimization Window must be adapted to each market as it is directly correlated with the market cycles or the "speed" at which a given market changes. For example, on the next picture we would be doing a Walk Forward Analysis using an "Optimization Window" size of 3 months and then an out-of-sample period of one month:
The result of a full Walk Forward Analysis test will be the sum of the results of all the smaller tests:
A strategy that is able to survive to a proper Walk Forward Analysis will show much higher levels of robustness than strategies trading traditional optimization parameter sets. Most curve fitted features or features added to improve short term results will not be able to survive this test as they will constantly fail during the different testing phases.
The goal of Walk Forward Analysis
There are two main goals to achieve with Walk Forward Analysis:
- Robustness: The main goal of Walk Forward Analysis is to achieve very high levels of robustness. By robustness I mean getting similar results in live trading and on backtests.
- Increased profitability: A proper Walk Forward process will allow the strategy to adapt itself to a changing market allowing it to trade more frequently and target more pips on favorable conditions and reduce trading frequency and targets on unfavorable conditions.
There are another secondary benefits:
- If the inefficience is erased from the market, a proper Walk Forward implementation will choose not to trade
- If then the inefficience returns to the market, it will slowly start to trade again
- Life of the strategy is much longer. The worst case scenario that would make us stop trading the strategy would be much tougher to reach. The elimination of the inefficience would be not enough for us to stop trading. In addition to that, the whole Walk Forward process must fail to detect this market conditions.