Should You Even Try to Beat the Market?

I should preface, beyond the normal disclaimer, this isn’t investment advice. I can’t prescribe investment advice because I don’t know you, and Composer isn’t in the business of offering advice in the first place. Rather, this piece is intended as a philosophical discussion in response to a question I get asked all the time: “Given most active managers underperform the S&P 500, why doesn't everyone just invest in index funds? Why does anyone trade actively? Should you even try to beat the market?”

There’s so much out there already in support of passive indexing that I won’t waste space rehashing the arguments against trying to beat the market. You can read Burton Malkiel’s A Random Walk Down Wall Street or join an entire community of Bogleheads, disciples of the godfather of passive investing and founder of Vanguard, Jack Bogle, if you want to be indoctrinated with the merits of passive investing. In contrast, there’s surprisingly little out there in defense of active trading. So instead, I’ll spend the rest of this article trying to make the best case for why active trading can make sense for some people, even if not for everyone (or even most people).

1. True passive investing doesn’t exist in the first place. Most people consider “buying the S&P 500” as a synonym for passive investing. Let’s break down what an S&P 500 index fund is: a market cap weighted index of 500 of the largest companies in the United States. But wait, which 500 companies? Why only the United States, and why market cap weighting? Well, the Standard and Poors’ committee decides which companies get included in the index, applying a variety of selection criteria. And they also decided that the correct way to weight members of the index is by market cap. This means the companies in the index change every couple weeks, and the weights of individual companies change constantly throughout the day in response to changes in market cap. The companies are decided by humans and the weights are adjusted programmatically. That sounds awfully active to me, and there’s no natural law that says S&P’s methodology is forever superior.

2. Index investing is susceptible to hindsight bias, too. The S&P 500 has had a really good run the last 15 years, with SPY generating a cumulative return of 595.7%, or 13.8% annualized, from May 2010 to the time of this writing (Source: Composer). As mentioned in the previous point, people use the S&P 500 as a proxy for passive investing, as if it’s obvious that any investor would have held SPY as their one and only investment if they had any sense in 2010. But this is hindsight bias to a much more alarming degree than most realize; what is obvious now really, really wasn’t 15 years ago. For example, EEM, an ETF representing the largest companies by market cap in emerging markets, absolutely smoked SPY from May 2003 to May 2010:

EEM generated a whopping 20.5% return annualized for this seven year time period, compared to SPY’s paltry return of 5% (Source: Composer). I have a crystal clear memory of pundits at the time saying that EEM’s outperformance was obvious, a logical certainty given that emerging economies were growing faster and had way more room to catch up with developed ones. If you had followed this “obvious” thesis in 2010, here’s what your returns would look like if you invested in EEM vs SPY:

Ouch. EEM returned only 3.4% annually for the next 15 years, a return closer to what we would expect from treasuries!

The Callan periodic table of investment returns shows that the top performing asset class changes a lot from year to year. It’s only obvious in hindsight which asset class will perform best, and there’s certainly no guarantee that the S&P 500 will outperform going forward - whether comparing to other passive indices or active strategies.

3. Passive investing depends on active market participants in order to work. There’s an ongoing debate about exactly what proportion of market participants need to remain active in order for markets to function and for passive investing to “work”, but no one debates that at least some active market participation is necessary. The good news is that the vast majority of daily trading volume is from active participants like asset managers, High Frequency Trading firms and market makers. Passive investing flows make up less than 5% of daily US equity trading volume, so we’re a long way from having to worry about markets ceasing to function.

But the fact that so much of daily trading volume is “active” is telling in itself; given how costly it is in terms of time, money and resources to actively trade, why would anyone bother if there was no potential reward? This is the core insight of Grossman & Stiglitz in their seminal paper, “On the Impossibility of Informationally Efficient Markets”, published back in 1980:

“We propose here a model in which there is an equilibrium degree of disequilibrium: prices reflect the information of informed individuals (arbitrageurs) but only partially, so that those who expend resources to obtain information do receive compensation.”

That’s a bit of a mouthful, but what they’re saying is that if markets were perfectly efficient, nobody would have an incentive to make them efficient through active trading, and then they would become inefficient again - so markets need to be a little inefficient all the time for the system to continue functioning.

Looking at the trading revenues of Jane Street, a prop trading firm that makes much of its money by arbitraging small discrepancies in the value of ETFs and other assets, it’s clear that providing liquidity and exploiting market inefficiencies is still handsomely rewarded:

Source: https://www.ft.com/content/24fea1d6-ba66-4b6b-814b-7bb72abfe58f

4. Hedge funds can actually do pretty well before fees. Warren Buffet famously wagered hedge fund-of-funds manager, Ted Siedes, that the S&P 500 would beat Siedes’ carefully selected group of hedge funds over the next decade. Buffet handily won the bet, with not one of the funds beating the S&P 500; Ted Seides officially conceded in 2017.

Efficient market proponents and index enthusiasts alike saw this as the final nail in the coffin for active management. After all, even the best in the world couldn’t beat the market after fees. And the thing is, they were right to dunk - hedge funds do mostly suck. But the reason for sucking matters here, and it’s because they charge such ludicrously high fees. From 1969 until 2025, hedge funds generated $3.7 Trillion in gains, but kept $1.8T for themselves in the form of fees, or nearly 49% of gross gains.

Multiple studies have looked at hedge fund performance before fees, and the consensus is that they do generate alpha - they just charge so much that investors in hedge funds don’t get a slice. Ibottson summarizes in his 2010 paper on hedge fund performance:

“We estimate a pre-fee return of 11.42%, which we split into a fee (3.78%), an alpha (3.01%), and a beta return (4.62%). The positive alpha is quite remarkable, since the mutual fund industry in aggregate does not produce alpha net of fees. The year by year results also show that alpha"s from hedge funds were positive during every year of the last decade, even through the recent financial crisis of 2008 and 2009.”

While that study ended in 2010, recent metanalysis arrives at a similar conclusion. The implication here is that if you can replicate hedge fund strategies without the fees, you could in theory beat the S&P in risk-adjusted returns.

5. The very top traders and funds are an existence proof that beating the market is possible. Rentech is forever heralded as the ultimate outlier in market-beating prowess, and for good reason. I’ve looked at this table of Rentech’s returns over and over and it never ceases to blow my mind:

At 66% annual returns before fees, I don’t think I’ve seen anyone argue this was luck or statistical fluke. And yet, I’ve seen a fair number of comments about how “no one beats the market”, and the existence of Rentech shows that’s simply not true. And it’s not just Rentech: Stanley Druckenmiller, George Soros, Ken Griffin and Paul Tudo Jones have all trounced the market over long time periods, even after fees.

Does this mean it’s likely for a random individual to deliver similar results? No, of course not. But improbable doesn’t mean impossible, and that’s a pretty important distinction. Most people won’t make it to the NBA, either, but doesn’t mean basketball isn’t a worthy pursuit, which brings me to the next point.

6. Trading can be really fun and intellectually stimulating (although not suitable for everyone) - whether as a career or as a hobby. Arguably the best reason to actively trade is that you find the process itself rewarding. Using the previous analogy, even if you don’t make it to the NBA, you’ll probably get into incredible shape and grow a lot in the process. Trying to beat the market is so hard it will stretch you to the limits of your intellectual abilities. You will learn about yourself, you will gain more awareness of your emotions, and you will learn how capitalism works at a very fundamental level.

If you get good enough, you can even secure a high paying career in finance. And even if you don’t work in finance, if you engage in something like quantitative trading, the skills are highly transferable to data science and engineering roles in tech.

Again, I’m not being prescriptive here. I can’t say if active trading is right for you, and for most people it probably isn’t. But I can say from personal experience it resonated with me and ultimately took me on an adventure of a lifetime, culminating in founding Composer. Some of my fondest memories are the periods in between jobs where I got to sit in a cafe and work on devising quantitative trading strategies. Something about staring at the markets and coding just puts me in a total flow state. Maybe I’ll never be Jim Simons, but I can say for sure I’ll never be Lebron, either.