Creating a Portfolio Allocation That Beats The Market

Creating a Portfolio Allocation That Beats The Market

I've always been a little more active than index investors and a little more passive than day traders. Sometimes I buy individual stocks that I expect to perform well within a 3-6 month period. Sometimes I just simply change my mind and want to try a different allocation.

My psychology lies somewhere between Eugene Fama's Efficient Market Hypothesis and the unknowing pessimism of Taleb. Some days I feel that a whole-market buy and hold strategy would be just fine. The next day I'm inspired and forced to reconsider when I read a new and insightful article or book.

The Problem With Buy and Hold

This strategy assumes the investor purchases a stock, fund, or other security and simply holds on to it through any changes in price. In theory, the market will always trend higher over time. Even if it drops 30% in a single year, two years later it's pretty likely to have bounced back up and gained even more.

However, the main issue here is that this style misses out on earnings. If I could sell the investment as it falls and then buy it again as it begins to rise, my overall returns would be hugely magnified.

If we could just time the market to avoid those downturns...

The Problem With Market Timing

Ah if it were so easy. Timing the market to buy and sell investments is the holy grail of investing. It's the pinnacle achievement that so few investors ever even come close to. In fact, the simple act of trying to time the market arguably makes one a trader and not an investor.

In study after study, academic article and empirical practice, trying to time the market simply results in losses.


Because often by the time investors sell a stock, the market is already starting to turn around. The market gives us false signals, human emotions get in the way, we hold the stock too long, we sell it at the wrong time, we buy just before a dip.

There are too many incalculable factors.

PLUS human irrationality is completely unpredictable. Investors tend to let emotions get in the way and we buy or sell at precisely the wrong time. A wave of selling in the market sparks panic and investors unload stocks all at once, causing the markets to fall.

You can see the problems of trying to time the markets. There are too many players, too many unpredictable moves. Too much risk.

Creating a Solution For My Portfolio

Over the last few days, I've been reading "The Quants: How a New Breed of Math Whizzes Conquered Wall Street and Nearly Destroyed It". The book takes a look at the rise of quantitative (math based) hedge funds and trading strategies on Wall Street. It also discusses the conflicting beliefs between arbitrage and Efficient Market Theory. It's a great read and it has kept me engaged.

My personal goal is to create the highest possible returns on my retirement portfolio. I'm currently a long way away from needing any of that money and so I'm quite comfortable taking on the volatility of the market.

That's why I like 100% equities. Greater return, greater volatility.

Concern About the Inflating Cost of Investments

However, I've been growing more concerned about the inflating value of the market overall. As of June 25th the S&P P/E ratio sits at 25.82.

P/E ratios are a quick way to gauge the relative value of stock - how cheap is it compared to how much the company earns. The lower the P/E ratio, the less money you're paying to participate in the earnings of that company. Low P/E ratios are generally regarded as a sign of a "discounted" stock.

Historically, however, P/E ratios of the 500 largest companies in the US stock market has been about 15.65. Historical S&P 500 P/E data gathered from

The market as a whole is currently 65% more expensive than the average dating back before 1880.

S&P 500 Average P/E Level by Year

As I look at the data and consider that the market is indeed more expensive than its historical average I'm hit with a conflicting view. As you can see in the chart above, the market has been above its average every year except 1995 and 2012 in recent memory.

So is it really a problem that P/E is currently higher than historical and growing? Well, in some ways yes. In some ways no.

Look at the returns for each year dating back to 1993 in the table below. Let's see if the returns of the S&P 500 have any visual correlation with P/E swings seen above.

S&P 500 Returns by Year

It appears at a glance that the spike in P/E of the market as a whole in 2002 may be a response to the decline in the returns of the market from 2000-2003. Again, in 2009 the P/E of the market as a whole shoots up, seemingly in response to the massive drop in the market of 2008.

My Hypothesis

At this point, I've become vaguely suspicious that there's something here I can use. Now, keep in mind that I'm not looking for a trading strategy. I'm looking for a portfolio allocation indicator.

Hypothesis: Increasing the defensive allocation of my portfolio in tandem with a rise in the P/E of the S&P 500 may result in better annualized returns.

The Test (Backtesting)

Now let's compare two portfolios to see if there's a difference in performance.

One portfolio (A) will be 100% index invested in the S&P 500. The second portfolio (B) will fluctuate based on my own pre-determined criteria which reduce the portfolios S&P exposure and replaces it with gold investments as the overall market P/E rises to certain levels.

Portfolio A (S&P Index Method)


I'm going to calculate returns on a $10,000 portfolio completely indexed into the S&P 500 starting January 1, 1998. A practical example of this would be the (SPY) EFT. This first portfolio is relatively simple to calculate.

This type of investment is often called an S&P Index "buy and hold" investment. It's well loved for its simplicity, low fees, and ease of use for any investor.

S&P Indexed Portfolio Growth '98 - '17

Portfolio A ends 2017 with a value of $44,584.07. This is an overall 445.84% gain. Annualized that's an 8.15% average return. Not bad, considering this portfolio endured both the Tech Bubble crash and the Great Recession!

Now let's compare it to my portfolio which adjusts into gold based on the levels of the S&P average P/E.

Portfolio B (P/E Weighted Gold Defensive)


Let me first explain the methodology of this strategy, and then show you performance.

By calculating the Mean (average) P/E of 15.65 and Standard Deviation (7.24) I was able to use light statistical guidelines to create an adjustable portfolio.

My goal setting out was to create a method by which I can enter and exit gold holdings based on expected upcoming crashes or market corrections. Reason being that I want to take advantage of a full S&P 500 allocation during "safe" times since the S&P tends to return very well during those times.

Note: Standard deviation is a statistics term which indicates the probability of data falling within a range of the mean (average) in a normally distributed set of data. The further the numbers get away from average, the less likely any given number is to appear.

So, for instance, if the average height of a human is 5' 8" then it is increasingly unlikely that any random human chosen will be of a certain height the further one "deviates" from that average. The likelihood of any random person being 6' tall is pretty high. The likelihood of someone being 7' 7" is extremely low.

In fact, 95% of all data in a normal distribution is expected to lie within +or- 2 SD from the average.

There are many caveats to applying these statistical analyses to financial markets and in order to increase your awareness of their inherent shortcomings, I highly recommend "Black Swan" by Taleb for your immediate reading.

Since the S&P often sees great returns above the P/E mean (15.65) and below the first SD (22.89) I decided that I want to fully participate in those returns.

I don't want to miss out on the great returns the market sees when the S&P P/E is below 22.89. In my mind, a P/E below this means the market is relatively within the normal historical price range and therefore "safe". For the purposes of my strategy and this approach, anyway.

For the purposes of this test, when average P/E ratios start to head above 22.89, I want to decrease my exposure to the S&P and increase my exposure to gold incrementally. My suspicion is that as P/E ratios approach 2 SD above the average (30.13) the probability of a market correction is significantly greater and the rewards of being fully invested in the S&P 500 are diminishing.

Essentially I wanted to create a sliding scale whereby my portfolio is as defensive as I want it to get by the time P/E levels exceed two standard deviations from the mean. I want to remain 100% aggressive when P/E are at or below 1 SD above the mean.

Therefore my allocation scale for this test will look like this:

Portfolio B Allocation Weight by P/E Level

As P/E levels increase, so too does the allocation to Gold.

At any P/E level below 22.8 the portfolio will be fully invested in an S&P Index strategy.

At any P/E level above 30.1 the portfolio will be 50% S&P Index and 50% Gold.

Why not allocate 100% of the portfolio to Gold when P/E levels reach 30.1? Because even at that level Gold does not always outperform the S&P. Plus, if you look at the charts above, the P/E level spikes tend to be trailing indicators of a major downturn. Meaning on the years the P/E levels are seemingly the highest, the S&P still sometimes returns well (26% return in 2009 when P/E was at 70!).

Why invest in gold when P/E gets abnormally high? Historically investors flock to seemingly safe and liquid assets such as gold which can be held and touched when markets go to hell. So, as P/E rises to dangerous levels, placing some of the portfolio into gold should theoretically reduce exposure to a stock market crash and take advantage of increases in the value of gold as investors buy in.

Basically, I'm simply trying to take advantage of some of the increase in gold sentiment during times of high P/E and market volatility. By allowing the P/E to dictate my move, I'm hoping to come up with a combination that outperforms a purely "buy-and-hold" system.

I will continue to test different P/E scales and allocations - this is simply the first test to see if my hypothesis holds water. Much refinement needed!


To pull this off, I needed to find historical annual return data for gold as an asset class. Since my backtesting goes back before gold ETF popularity, I had to search for data outside the ETF world. I found my data at

Returns for the Gold Asset Class and the S&P 500

Note that gold returns for 2016 and YTD 2017 are taken from the return of (IAU).


Overall the dynamically weighted portfolio (B) outperformed the 100% S&P portfolio by a surprising margin. The following is a graph of Portfolio B versus Portfolio A (indexed).

S&P Index Portfolio (A) vs Dynamically Weighted Portfolio (B)

Overall Portfolio B, the P/E weighted dynamic portfolio beat the S&P Indexed portfolio by 15.9%. Had you invested $10,000 into Portfolio B in 1998, it would be worth $51,652 today. That's $7,068 more than Portfolio A over the same period.

Did it outperform every year? NO!

From 1998 - 2001 the dynamic portfolio underperformed due to allocation into gold which lagged the returns of the S&P. Thanks to an increasing P/E just before the Tech Bubble burst in 2002, Portfolio B was in a position to take advantage of gold's 23.96% gain that year, while the S&P dropped 22.1%.

Flaws In The Test

Immediately I am aware of several glaring flaws in the results of this test.

First, the test is run using annual returns and P/E ratios that are only knowable after the fact. For this reason it would seem prudent to use the P/E ratio of the prior year to determine next year's allocation.

For instance, on Dec 31 of 1998 I would use the P/E of that year to determine my allocation for Jan 1, 1999.

Functionally, however, this simply offsets the allocation by a year. The impact of this shift may actually be positive as it could cause the portfolio to begin buying gold even earlier, at cheaper price, though it may also cause the portfolio to sell gold too soon.

Much like problem #3, some of this issue can be solved by using a 3 or 6 month allocation rebalancing instead of annual.

Second, the allocations are very binary. It's either S&P index, or gold. More likely I would use a defensive portfolio of gold, utilities, consumer staples, bonds, etc. This would, for instance, decrease sector risk overall and help hedge against the risk of buying in to real estate, which is historically stable, just to find out that real estate is the cause of the decline (2008).

Third, the portfolio would be more responsive to market corrections and downturns were it to be reallocated on a 6 or 3 month basis. This would increase trading costs, but also increase the responsiveness of the portfolio allocation to changes in market conditions.

So, What?

If you managed to stick with me this far, you're probably asking, "So what?" Unless, of course, you've already grasped at the potential ramifications of this strategy.

So what is that the P/E weighted strategy outperforms a buy-and-hold strategy by 15.9% over a 19-year period including two of the biggest modern crashes!

According to yCharts, the annual S&P return historically is 11.72%. Over 40 years a retirement account starting at $10,000 would be worth $841,854 invested entirely into the S&P 500 index. A retirement account using the dynamic gold weighting strategy would be worth $975,709.

One simple portfolio change with the potential to pay you back $133,855 at retirement. What would you do with that extra money?

How to Use Correlation to Improve Your Investing

How to Use Correlation to Improve Your Investing

Acorns vs Stash - Investment Apps Comparison

Acorns vs Stash - Investment Apps Comparison