A Quanty Way To Analyze Sentiment, Momentum and Breadth in Equities

Throughout the day, the financial media reports on price developments in the stock market, among others. But, they only focus on stock indices and any stocks that are part of the day’s news, such as those reporting earnings that day. It is like standing on the edge of a pond all day, and taking note of the changes on the surface of the water, and any creatures that pop out of the water; there’s a whole world of things beneath the surface that you are not privy to.

I am not critiquing the financial press; I’m just saying that if you want to know what’s really going on with stocks—like what the market breadth is like, what is driving the index returns, what is the sentiment and momentum like, and more—you will need to dig around some more for that information. The weekly market report published in this newsletter is one of the places you can look to for that kind of analysis.

One of the things I want to include in that report is a section about this kind of stuff. While trying to figure out how to go about it, I decided to do some backtests of the various indicators I will be using so that I know what to include and what not to, and to learn what to look out for when using these indicators.

There are many indicators used to analyze the underlying dynamics of an index. My plan for the weekly market report is to start small and then add or remove them depending on their proven usefulness. In this post, I focus on two main indicators of sentiment and momentum, and market breadth.

What I Did

One of the reasons you will not find much commentary or research on what index members are doing is that keeping track of index members isn’t easy. Some paid services offer this information, but subscription prices can be high. One way to find out is to do a bit of web scraping. In this case, the web scraping involved going to the S&P 500 Wikipedia page, getting the current members, getting all the added and removed stocks, and reconstructing the historical constituents. I used R for that.

Due to de-listings, mergers, and acquisitions, there were a lot of missing tickers when I went to download price data. In 1990 (how far back I went), there were 87 and in Jan 2021, there were 24. I decided that 24 was not that bad and that any more would affect the strength of the indicators, so I started the backtests from Jan 2021 to Oct 2024. Is it long enough for the results to be statistically meaningful? I don’t know, but I would rather have a short period when the indicator signals are reliable than a long one when they are not. Plus, the objective here is not to get hard and fast rules to apply to these indicators but to understand how they work. With the list of historical constituents and the corresponding historical prices, I calculated the percentage of constituents above their 50-DMA, 100-DMA, and 200-DMA, and the number of constituents outperforming the index.

What I like about the first indicator is that it does not rely on the actual returns of the index components. It tells you something about sentiment and momentum in stocks in a way that is comparable over arbitrary periods, and robust to outliers. It also says something about market breadth. The second one is good for market breadth but is influenced by outliers.

I’ve seen some analysts use indicators like average distance from a DMA (how much a stock needs to move to reach a daily moving average) and the average distance from the index (how much a stock needs to gain to catch up to the index’s returns). These indicators are sensitive to outliers, and averages murk the picture; we are searching for clarity. Two indicators that I think are also good and will focus on later on are the percentage of stocks in a bull/bear market (number of stocks in a new 52-week high/low) and advance/decline (percentage of stocks going up or down).

Editor’s note: I’ve been working on a comprehensive backtest of CFTC data. It got too big that it wasn’t feasible for me to write the results, so I decided to publish them on a web app. I can’t afford to hire help rn so I have to do it all myself and that’s why it’s taking forever. But it’s worth the wait :)

For the backtests, the goal was to see if the indicators gave a reliable signal about the index’s direction; not to outperform the benchmark (buy-and-hold). Therefore, there are backtesting best practices I didn’t follow including transaction fees and stop losses. I’ll also say that there is a small amount of survivorship bias here because the stocks that were used to calculate the indicators are the ones that are still around today, but from 2021, the number of missing stocks kept declining to about 7 in Q4 2022 which I think reduces this bias. These are the important things to remember when you look at the backtest results.

Overview of the Selected Indicators

As I said, the two indicators we focus on are market breadth, and sentiment and momentum in stocks. Market breadth is the number of index components participating in its advance or decline. A naive way to calculate this is by just tallying the stocks going up or down and checking if the index went up or down too, but indices are weighted averages so this might not be the best approach. A better way is to look at the number of stocks outperforming the index. These are the stocks that are in the driver’s seat. Breadth widens when more stocks outperform the index, and contracts when fewer stocks do. However, underperformers can significantly influence this indicator because if there are a few big losers, they will pull the index down and many other members will outperform it. That’s why you should look at several indicators together to get the full picture.

Sentiment and momentum don’t have to go together—they are different things! Sentiment is how investors feel about the market (bullish/bearish) and momentum is about the strength of a trend. For more assets, it is possible for investors to be bearish and stocks to have bullish momentum, and vice versa. But in this case, the percentage of stocks above a moving average gives some information about the overall sentiment and momentum because you are aggregating over all index members.

Percentage of Constituents Above a Moving Average

You would think this indicator is lagging because it involves moving averages, but it isn’t. It moves with the index. How much it moves gives you a signal about the sentiment and momentum in stocks. Here we have the S&P 500 this year, and the percentage of constituents above the 50, 100, and 200 DMAs.

The percentage of stocks above their 50-dma indicator is more volatile which is expected. The percentage of stocks above their 100-dma moves within a narrower band but it moves up and down. The percentage of stocks above their 200-dma has been quite high all year, spending most of the year above 70%.

Here is the same chart going all the way back to November 2021.

Most of the time when the percentage of stocks above their 50-DMA went above 80%, it would top out and the market would turn. However, in January 2024, this didn’t happen. The percentage of stocks above their 50-DMA topped out at 92.5% on January 2nd, and then dropped and have never been to those levels since, even though the market has seen going higher and higher. This was the bullish sentiment extreme of the year after stocks started rising in October 2023 in anticipation of rate cuts. All three indicators rose from their yearly lows from Oct ‘23 to Jan ‘24, however, the number of stocks above their 200-DMA remained high all year which shows that momentum was bullish.

So, it seems that they all capture slightly different things. The percentage of stocks above their 50-DMA is more of a sentiment indicator; when it reaches extreme levels, a turnaround is usually imminent, but not if there is momentum to the opposite side which is captured more by the other two—especially the percentage of stocks above their 200-DMA.

You can see this if you look at the correlation of the indicators and the prices. The percentage of stocks above their 50 and 100 DMAs have a correlation of 99%. They are the same but one moves more dramatically and both have a weak positive correlation to prices—both are more sentiment than momentum indicators. The percentage of stocks above 200-DMA has a higher correlation to the prices than the other two and a higher (and equal) correlation to the other two indicators than prices—it is more of a momentum than a sentiment indicator.

Percentage of Outperforming Constituents

This indicator is difficult to read because it varies wildly from one trading day to the next. Here is the chart along with S&P 500 prices.

If you zoom in on the Oct ‘23 to April ‘24 rally, here is how it looks.

Not readable. Compare that to the percentage of stocks above their DMA over the same period below.

Despite the percentage of outperforming constituents indicator not being too readable, backtest results show that it gives good signals at the extremes that are more timely than the later set of indicators (1-3 days before the market turn).

Backtests

I backtested five strategies. Four were directional bets when the indicator crossed over a long threshold (low threshold i.e. 20%, the indicator has to first go below the threshold and then come back above it) or crossed under a short threshold (high threshold i.e. 80%). The fifth strategy was when the percentage of stocks above 50-DMA crossed the percentage of stocks above 200-DMA (when sentiment matched momentum).

For each, I defined a set of thresholds and then backtested all their combinations. I also isolated the long-only, short-only, and long-and-short results. Let’s go over them.

Strategy 1: Use Thresholds on Proportion Above 50-DMAs to Time Entries

In this strategy, the long thresholds were 10-35 and the short thresholds were 70-95, in steps of 5.

Long Only

Here, the entry signals were the long threshold crossovers and the exit signals were the short threshold cross-unders, but no short positions were taken, instead, longs were closed at the short entry signal. Here are the results in a matrix.

Over the same period, a buy-and-hold returned 25.6% therefore going long only, regardless of the combination of entry and exit signals, was net positive. However, 14/30 of the combinations outperformed the buy-and-hold (without fees and stop losses). This shows that you got better entries and exits using 14/30 combinations if you had a long bias. There’s a pattern where regardless of the entry threshold, the higher the exit threshold, the better the returns. This means that waiting for extremely bullish sentiment to exit your position was more important than when you went long. The reason for this is that this indicator moves quite fast so the entry thresholds are all crossed in a short time, but it takes a while to get to the other extreme, which usually coincides with a market top.

The table below gives the metrics of the best and worst combinations.

3-5 trades are taken in each combination of signals and the win rate is between 60-100%. Again, this is due to the long-only bias of the market during this period. It is interesting to note that despite this, the max drawdown period is quite high. One of the two second-best combinations had a max drawdown duration of 293 days.

Short-Only

The short-only version, where the signals are reversed, doesn’t do anywhere nearly as well. The matrix below shows why. The exit signals for this strategy are almost never triggered because the market keeps going up.

The interesting thing is that despite this, the combinations have win rates of between 50 and 66.7%.

Long and Short

Here are the returns of taking both long and short positions. A matrix like this one can show you how robust a strategy is. Here we see that if the long threshold is below 20, the results are bad, but if the exit threshold is 75, then the results are quite good regardless of the entry threshold. Also, long thresholds from 30-35 do well despite the exit threshold. However, this is overfitting; a different timeframe will probably give different results. What is important to note is that waiting until sentiment, momentum and breadth starts to pick up to go long then waiting for the other extreme to go short is profitable.

Even though I haven’t included transaction costs, I don’t think they affect these results that much because you only need to trade on asset (SPY) and there are 6-13 trades in this about 3 years.

Strategy 2: Use Thresholds on Proportion Above 100-DMAs to Time Entries

The second strategy involved doing the same thing as the first but using the percentage of stocks above their 100-DMA. It is 99% correlated to the proportion of stocks above their 50-DMA but isn’t as volatile which means it is less sensitive to short-term changes—it has more powerful signals.

Long Only

Here are the results for long-only positions. 11/24 outperformed the buy-and-hold return of 25.6%. This is again due to the market going mostly up during the period. However, the best-performing combinations here did better than those of the same indicator using 50-DMA (strategy 1).

You also see that the entry signal didn’t matter so much apart from the 10% threshold (because this indicator moves around a narrower range), but the higher the exit threshold, the better the results. The point here is that this indicator gives more powerful signals than the one with 50-DMA.

Another great thing about this indicator over the latter is that the max drawdowns are lower. Here, the second-best strategy had a max drawdown of 16% while one of the second-best strategies of the latter had a max drawdown of 20%.

Short Only

Even the short-only positions are better than in the latter indicator. Here the best combination had 8% and in the latter, it had 4%. The proportion of combinations with sub-20% returns is also lower. However, shorting was generally a bad strategy regardless of the indicator.

Interestingly, the max drawdown duration of this indicator is better than the latter one with 50 DMAs—231 days compared to 434. This is again because the 100-DMA gives a more powerful signal than the latter due to not being so short-sighted.

Long and Short

The results of the long and short show just how robust this indicator is. A long-entry threshold of 10% is too low since this indicator rarely gets to that level, so it has poor results. Since it also serves as the short-exit signal, in a bull market, it is less likely to be triggered. A short entry signal of 50% is also poor since in a bull market, you would be cutting your winners too soon, and entering your shorts too late. But other combinations all give positive results. Interestingly, the best-performing combination was a long entry/short exit at 35% and a short entry/long exit at 55%. This is a narrow range, but this indicator moves slowly. With this combination, you would enter markets late and exit late always (since the long exit is triggered by crossing under the threshold, meaning it must first go over it and then pull back).

Here we see that the best strategy, despite using a narrow range, led to 19 trades over 3 years with a win rate of 77.8%. The max drawdown was 17.6%.

Strategy 3: Use Thresholds on Proportion Above 200-DMAs to Time Entries

Here we do the thing again but with the 200-DMA version of the indicator. We’ve seen that for most of this year, over 70%of stocks have been above their 200-DMA. This shows that this indicator has more to do with momentum than sentiment. If you want to check this analytically, you can create a momentum factor and then regress it against, or calculate its correlation to this indicator. However, while you would expect this indicator to be late, it isn’t but it takes a lot for it to move very far.

Long Only

Like the other two similar indicators, for the most part, the long-only strategy has positive results. But here we see a combination that gave negative results. We also see that this indicator gives the most powerful signal since it is quite robust. You can tell this because similar results are organized in clusters and patterns. For instance, long entries that used 10% broke even regardless of the exit; exiting at 80% short threshold (exiting too soon) gave decent returns but couldn’t outperform the benchmark; and entries between 25-35% with an exit at 60% performed about the same, and outperformed the benchmark.

This strategy however had few trades and very long max drawdown periods (sometimes 2/3 of the backtest period). However, the max drawdowns of the two best combinations were both 16.9%.

Short Only

For the short-only strategy, we see similar patterns of which combinations resulted in good or poor results. The best results are when the short entry is 60% and the exits are 25-35%, but the results are quite low compared to the benchmark. Still, the best result is 8% which equals that of the 100-DMA version of the indicator and is higher than the 3% of the 50-DMA version.

While the best performing combination here is similar to the 100-dma version, the max drawdown is worse at 19.4%.

Long and Short

The long and short strategy shows that only long entry/short exit of 25-35% and short entry/long exits of 60% outperform the benchmark—only 3 of the remaining 27 combinations have positive results. However, the returns of the combinations outperforming the benchmark return are about twice the benchmark’s.

When I first looked at these results, I was worried that I had overfitted to the point that only one trade was taken and it happened to be the perfect trade in hindsight, but no. The best-performing combination has a return of 51% and had 6 total trades; 5 closed and one still active, up 38.9%. It had a win rate of 60%. However, the max drawdown was higher than the long-only and short-only at 22.4%.

Strategy 4: Percentage of Outperforming Constituents

Remember earlier when I showed you the chart of the percentage of outperforming constituents and it looked messy? Well, it turns out to be a poor indicator—more noise than signal—even though the results here show that it can outperform a buy-and-hold.

Long Only

The main takeaway from this matrix is that you want to go long when at least 20% of the constituents are outperforming the index and exit when not more than 90% are outperforming the index. The 12 combinations meeting this criteria had positive results—5 outperformed the benchmark. Any longs taken when less than 15% of the stocks outperformed the index led to breakeven, and any exits executed when less than 75% of the stocks outperformed the index led to suboptimal returns.

Here we see that the best combination only had one trade, which is still open. The second-best had two trades, one of which is still open.

Let’s look at when these trades happened. The chart below shows the best combo of signals. It entered a trade on Aug 22nd when the number of stocks outperforming the index crossed above 25 for the first time in a long time.

On August 21st, the percentage of stocks outperforming the S&P 500 fell to 21 and then on August 22nd, it went to 48. However, this indicator is messy because it fluctuates wildly and is sensitive to outliers. Here is how that period looks on a chart.

But August 22nd was a local bottom so it is worth paying attention to, I guess.

Short Only

For the short-only strategy, we have the same problems; only 1 trade was executed by the best combination of signals. By now it’s evident that short-only positions don’t do well (duh!, it’s a bull market), so I won’t talk about the other combinations.

The one short was triggered on July 24th for 5615. The SPX then made a local top at 5667 then dropped 9.27% by 5th August. The exit signals, when only 20% of the stocks outperformed the S&P 500, led to a premature exit 3 days before the bottom. This shows that the indicator is not all that reliable for signals but is worth paying attention to perhaps in combination with other ones.

Here are the corresponding signals. On July 11th about 89% of the constituents outperformed the index. This dropped to 56% on July 12th. Then on July 31st, only 21% of the constituents outperformed, which rose to 47% on August 2nd. Perhaps a moving average or some signal processing techniques can improve this indicator, idk.

Long and Short

Going both long and short led to better results but the number of trades is still quite low. The best-performing strategy had 3 trades, and the second-best had one that was still active at the end of the backtest period. The max drawdown is also quite high at 25.4%.

Strategy 5: Proportion of Members Over Their 50-DMAs Crossing Proportion of Members Over Their 200-DMAs

The final strategy involved crosses of the percentage of stocks over 50-dma crossing the percentage of stocks over 200-dma. There were no parameters in this strategy and no version of it, long-only, short-only or long-and-short, outperformed the benchmark. However, a better indicator than this is the percentage of stocks whose 50-dma is over its 200-dma. That might capture what I had thought this indicator would.

Conclusions

The results show that it is insightful to analyze equity index cross-sectional indicators, and it explains how to do that. The percentage of constituents above their 50, 100, and 200-DMA indicators do a good job of measuring the sentiment and momentum of the stock market, and their breadth, to varying degrees. The 50-DMA version is more about sentiment but can also be misleading since sentiment and momentum are different and don’t have to move together. Plus sentiment changes fast. The 100-DMA version has superior returns to the 50-DMA version and captures sentiment and momentum in a more balanced way than the latter. The 200-200-DMAdma version is more about momentum and has the highest returns. It is only triggered when things are starting to get going.

They are quite timely and also unbiased—they are sensitive to outliers. Here is how they stand at the moment.

The percentage of outperforming constituents is interesting because at some market turns, it gets triggered right before the top or bottom. That is, the extremes of this indicator happen right before the top or bottom, by a few days (1-3 days). However, it fluctuates dramatically and is therefore mostly noise with few signals. But when it gives a signal, it is reliable and timely.

Supplementary Material:

If you want to run the code for this post, you will need this dataset of constituents.