What Actually Predicts The Stock Market?

(Not much)

Anyone who has spent much time with a quantitative tool trying to regress data points against future returns already knows: nearly nothing works.

It also seems to be substantially easier (read: still quite difficult) to find relationships which predict returns over an investable time frame.

The best fits seem to be over 10 or 15 years. This: 

  1. reduces the sample size, making the veracity of the regression demonstrating causation dubious
  2. makes it very  difficult to forward test
  3. even with the tightest past explanatory power, does not account for the massive developments and shifts in behaviour which occur on a regular basis

The surest way of making a fool of yourself is to predict the future.  The folly of prediction probably has an exponential relationship with expanding time frames.

With that said, I present two compelling long-run regressions.

First up is the “M/O” ratio (the ratio of the middle-age cohort, age 40–49, to the old-age cohort, age 60–69), as defined in the San Fransisco Fed paper, regressed against the P/E ratio:


It is an explainable relationship:  it evidences that marginal asset preference is demographically driven.  And that we can certainly further rationalise: our demands for return and safety (which are usually mutually exclusive), as well as our income, are reasonably predictable.  The middle-aged cohort has the most income to invest, but not yet the preference for capital preservation that retirees and the soon-to-retire cohort demonstrate.

Some criticisms I’d level at this explanation:

  • It only explains one asset class out of many in the marginal asset preference framework.  The perceived value and safety of all of the possible asset classes plays into preferences.
  • The model is inconsiderate of institutional (ex-pension funds) and foreign money whose behaviour is not explainable by U.S. demographics.
  • Cyclical variation is, and could be, massive.

An important determinant of future returns is the starting value which you purchase:


With an R of 0.54, there is plenty of room for alternate explanatory factors, but buying cheap has doubtlessly provided support for future returns.  The absolute attractiveness of an asset class should drive future preference.  But this method naturally suffers from the same vacuous assumption that the characteristics of competing asset classes do not affect future preferences.

Perhaps the best fit over any time-frame for any variable against the S&P 500 is my cyclical adjusted EY


Busigin’s Cyclically Adjusted EY is calculated as:

S&P 500 TTM EY – (Corporate Profits / GDP) * 100

The first two criticisms aimed at the aforementioned M/O-P/E model are also relevant to this model, and brings some of its own explanatory weaknesses:

  • The composition of US corporate profits are partially acyclical – the internal income distribution is usually (if not itself erroneously) viewed as cyclical/mean-reverting, but the structural shift towards foreign profits is probably not so.

In order to side-step the pitfalls of very-long-run prediction, we step down to predicting the next two years:


I posit that the ratio represents cyclical forces on the marginal asset preference framework. The mean-reverting preference for liquidity has typically shifted back into return-seeking.  It additionally illustrates the portfolio effect from Fed easing:  mechanically rebalanced portfolios automatically shift out Treasury gains into equities (and vice versa).

Stepping closer to nearer-term prediction, the 2s10s Treasury curve spread has demonstrated a great deal of predictive power over equity returns — y/y, lagged 5 quarters:


With an R of 0.57, it demonstrates the kind of inflation expectations that move both the Treasury and equity markets.  While the previous forecasting metrics focus on valuation, the Treasury curve is largely a cyclical measurement.

We can go back further from 1998, and the fit visually looks good, but mathematically declines:


The relationship clearly holds, but the degree to which it leads equity returns is clearly non-stationary.  It is probably then appropriate to discard the assumption of stationary lead-time moving forward.

Business Insider Emails & Alerts

Site highlights each day to your inbox.

Follow Business Insider Australia on Facebook, Twitter, LinkedIn, and Instagram.