How to Avoid Overfit Investment Strategies
What is an overfit investment strategy?
Overfitting refers to developing an investment strategy that performs well in a backtest because parameters are optimized for that specific time period.
Backtesting is a method of evaluating an investment strategy by looking at how it would have performed in historical market conditions. Unfortunately, there are several biases that can lead investors to overfit strategies, including look-ahead bias and data-snooping.
Overfitting is a concern because strategies that are optimized for a historical period are unlikely to be successful or deliver the same kind of results in future periods. For example, an investment strategy could be designed to do exceptionally well during the COVID crash of 2020, avoiding drawdowns and rebounding quickly with the market. However, it is unlikely that future crashes will have the same leading indicators or timing of drawdowns.
An example of overfitting
Consider an investor evaluating a tactical overweight to value or growth stocks to outperform the S&P 500 Index. To evaluate this strategy, they backtest the performance of Vanguard’s Growth ETF (VUG) and compare it to the S&P500 (SPY) and Vanguard’s Value ETF (VTV). Going back to the beginning of 2005 through 2021, it looks like tilting toward growth has been an excellent strategy. Growth (VUG) has returned almost 2% more per year than SPY and more than 3% more per year than VTV.
This is an example of look-ahead bias because a well-informed investor would know that growth stocks, including Amazon, Google, Apple, and Meta outperformed over this period. The essential question to answer is whether growth will continue to outperform. Further, will the factors that led to growth’s outperformance over the period continue, and then based on that, should investors have conviction in this strategy?
Another, and somewhat related issue is data snooping. This bias involves testing multiple strategies and picking the best one. Investors get hooked on tweaking and optimizing their backtest results, which leads to overfitting.
How to avoid overfitting strategies
To effectively use backtesting and avoid overfitting, investors should consider its output as data, not returns. Here are a few things investors can do to improve their strategy creation process.
1. Start with a purpose
Investors should start with a clear rationale for their investment strategy. That means asking why they are building this strategy and why it will work.
For example, to reduce portfolio drawdowns, investors may consider inverse volatility weighting or dynamically shifting from risky assets (e.g., stocks) to less risky assets (e.g., treasuries). These strategies have a clear, intuitive explanation for why they would achieve reduced volatility and lower drawdowns.
2. Be forward-looking
If the only answer for why a strategy will be successful is because it worked in the past, investors should re-evaluate. Being forward-looking requires critical thought. Will the environment or reasons for a strategy’s past performance continue?
Consider a risk on risk off trade that shifts from US equities to treasuries based on drawdowns in the S&P 500. Should investors expect the strategy to reduce volatility and drawdowns? Yes, that’s reasonable, given the strategy shifts to US treasuries, an asset with lower historical and expected standard deviation. Should investors expect the strategy to outperform the S&P 500 or perfectly time downturns? No, that’s not reasonable. Future market movements, including the timing, length, and depth of drawdowns, have proven almost impossible to predict.
3. Test multiple time periods
Some experts recommend using the longest time-period possible to evaluate an investment strategy. Viewing performance across many different market environments (e.g., high rates, low rates, recession, expansion) should paint a clearer picture than performance over the last month.
However, it is important to zoom in and test the strategy in different time periods. How did the strategy perform through the GFC? What about the COVID Crash? Or how about 2021, when volatility was muted, and the S&P returned ~27%? By zooming in, you can start to draw insight into when the strategy performs well and when it may lag. Take it a step further and pick random time periods. Or, roll through backtests starting with the earliest period and move forward by set intervals, a year or a month, for example. Evaluating performance across all these periods will give you insight into how the strategy may perform when it’s live in an uncertain future state.
4. Evaluate trading frequency
Strategies must balance responsiveness and the risk of overfitting. More frequent trading, like daily and weekly rebalancing, and some threshold-based strategies, are more responsive to changing market conditions. However, the more frequently a strategy trades, the greater chance it is overfit. That does not mean all frequently traded strategies are inherently bad, it simply means there is a greater risk of overfitting for these strategies as compared to monthly or quarterly rebalanced strategies.
Further, every time a strategy trades, it incurs transaction costs, namely bid-ask spreads. In addition, transactions can trigger capital gains taxes. And short-term capital gains, where assets are held for less than a year, are taxed at a higher rate than long-term capital gains. Both of these factors contribute to the unfortunate truth that higher turnover = higher cost.
These higher costs and the potential for overfitting should be weighed against the benefit of having a more responsive strategy. For this reason, the CFA institute recommends starting with monthly rebalancing for backtests.
5. Adjust strategy parameters
To avoid overfitting, test your strategy using a range of inputs, checking different levels of the same parameter. For example, use 5%, 7%, 10%, and 15% drawdowns as a parameter for a strategy. Are the results consistent? Does one result look better than the others? If so, it may be statistical noise. There likely isn’t a good argument why using a drawdown of 7% would be significantly better than 5% or 10%. A good strategy should be robust to small changes in parameters.
6. Use the reality test
It is always helpful to sanity check results. If returns are well above the S&P 500 and Nasdaq 100 without commiserate risk (e.g., similar drawdowns), the strategy may be overfit. If a strategy perfectly times a market downturn and rebound, it may be overfit.
It is essential for investors to consider the risks to their portfolio if trades are timed incorrectly because even the best investors and strategies are wrong at times. If investors are using leveraged, inverse, or volatility-linked ETFs, gains and losses can be amplified.
Investing in securities involves risks, including the risk of loss, including principal. Composer Technologies Inc., is an SEC Registered RIA. The SEC has not approved this message.
Certain information contained in here has been obtained from third-party sources. While taken from sources believed to be reliable, Composer has not independently verified such information and makes no representations about the accuracy of the information or its appropriateness for a given situation. In addition, this content may include third-party advertisements; Composer has not reviewed such advertisements and does not endorse any advertising content contained therein.
This content is provided for informational purposes only, as it was prepared without regard to any specific objectives, or financial circumstances, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities or digital assets are for illustrative purposes only and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not intended as a recommendation to purchase or sell any security and performance of certain hypothetical scenarios described herein is not necessarily indicative of actual results. Any investments referred to, or described are not representative of all investments in strategies managed by Composer, and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results.
Charts and graphs provided within are for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see Composer's Legal Page for additional important information.