How to Avoid Overfit Investment Strategies

What is an overfit investment strategy?

Overfitting refers to developing an investment strategy that performs well in a backtest because parameters are optimized for that specific time period.

Backtesting is a method of evaluating an investment strategy by looking at how it would have performed in historical market conditions. Unfortunately, there are several biases that can lead investors to overfit strategies, including look-ahead bias and data-snooping.

Overfitting is a concern because strategies that are optimized for a historical period are unlikely to be successful or deliver the same kind of results in future periods. For example, an investment strategy could be designed to do exceptionally well during the COVID crash of 2020, avoiding drawdowns and rebounding quickly with the market. However, it is unlikely that future crashes will have the same leading indicators or timing of drawdowns.

An example of overfitting 

Consider an investor evaluating a tactical overweight to value or growth stocks to outperform the S&P 500 Index. To evaluate this strategy, they backtest the performance of Vanguard’s Growth ETF (VUG) and compare it to the S&P500 (SPY) and Vanguard’s Value ETF (VTV). Going back to the beginning of 2005 through 2021, it looks like tilting toward growth has been an excellent strategy. Growth (VUG) has returned almost 2% more per year than SPY and more than 3% more per year than VTV.

This is an example of look-ahead bias because a well-informed investor would know that growth stocks, including Amazon, Google, Apple, and Meta outperformed over this period. The essential question to answer is whether growth will continue to outperform. Further, will the factors that led to growth’s outperformance over the period continue, and then based on that, should investors have conviction in this strategy?

Another, and somewhat related issue is data snooping. This bias involves testing multiple strategies and picking the best one. Investors get hooked on tweaking and optimizing their backtest results, which leads to overfitting.

How to avoid overfitting strategies

To effectively use backtesting and avoid overfitting, investors should consider its output as data, not returns. Here are a few things investors can do to improve their strategy creation process.

1‍. Start with a purpose

Investors should start with a clear rationale for their investment strategy. That means asking why they are building this strategy and why it will work. 

For example, to reduce portfolio drawdowns, investors may consider inverse volatility weighting or dynamically shifting from risky assets (e.g., stocks) to less risky assets (e.g., treasuries). These strategies have a clear, intuitive explanation for why they would achieve reduced volatility and lower drawdowns.

2. Be forward-looking

If the only answer for why a strategy will be successful is because it worked in the past, investors should re-evaluate. Being forward-looking requires critical thought. Will the environment or reasons for a strategy’s past performance continue? 

Consider a risk on risk off trade that shifts from US equities to treasuries based on drawdowns in the S&P 500. Should investors expect the strategy to reduce volatility and drawdowns? Yes, that’s reasonable, given the strategy shifts to US treasuries, an asset with lower historical and expected standard deviation. Should investors expect the strategy to outperform the S&P 500 or perfectly time downturns? No, that’s not reasonable. Future market movements, including the timing, length, and depth of drawdowns, have proven almost impossible to predict. 

‍3. Test multiple time periods

Some experts recommend using the longest time-period possible to evaluate an investment strategy. Viewing performance across many different market environments (e.g., high rates, low rates, recession, expansion) should paint a clearer picture than performance over the last month.

However, it is important to zoom in and test the strategy in different time periods. How did the strategy perform through the GFC? What about the COVID Crash? Or how about 2021, when volatility was muted, and the S&P returned ~27%? By zooming in, you can start to draw insight into when the strategy performs well and when it may lag. Take it a step further and pick random time periods. Or, roll through backtests starting with the earliest period and move forward by set intervals, a year or a month, for example. Evaluating performance across all these periods will give you insight into how the strategy may perform when it’s live in an uncertain future state. 

4‍. Evaluate trading frequency

Strategies must balance responsiveness and the risk of overfitting. More frequent trading, like daily and weekly rebalancing, and some threshold-based strategies, are more responsive to changing market conditions. However, the more frequently a strategy trades, the greater chance it is overfit. That does not mean all frequently traded strategies are inherently bad, it simply means there is a greater risk of overfitting for these strategies as compared to monthly or quarterly rebalanced strategies.

Further, every time a strategy trades, it incurs transaction costs, namely bid-ask spreads. In addition, transactions can trigger capital gains taxes. And short-term capital gains, where assets are held for less than a year, are taxed at a higher rate than long-term capital gains. Both of these factors contribute to the unfortunate truth that higher turnover = higher cost. 

These higher costs and the potential for overfitting should be weighed against the benefit of having a more responsive strategy. For this reason, the CFA institute recommends starting with monthly rebalancing for backtests.

5. Adjust strategy parameters  ‍

To avoid overfitting, test your strategy using a range of inputs, checking different levels of the same parameter. For example, use 5%, 7%, 10%, and 15% drawdowns as a parameter for a strategy. Are the results consistent? Does one result look better than the others? If so, it may be statistical noise. There likely isn’t a good argument why using a drawdown of 7% would be significantly better than 5% or 10%. A good strategy should be robust to small changes in parameters.

6. Use the reality test

It is always helpful to sanity check results. If returns are well above the S&P 500 and Nasdaq 100 without commiserate risk (e.g., similar drawdowns), the strategy may be overfit. If a strategy perfectly times a market downturn and rebound, it may be overfit. 

It is essential for investors to consider the risks to their portfolio if trades are timed incorrectly because even the best investors and strategies are wrong at times. If investors are using leveraged, inverse, or volatility-linked ETFs, gains and losses can be amplified.