There has been lots of discussion at Portfolio123 regarding prediction of stock model performance out of sample (post-launch) based on factors extracted from past data. I thought that I would take the time to add my own analysis.
Most of the analytical work has involved something called Gain Stock per Day (GSD). Some of the problems I have noticed with the previous work is the focus on (pre-occupation with) GSD and the prediction (annual excess return).
Some of the problems with the analysis include inconsistent pre-launch and post-launch time periods and use of look ahead data (GSD includes post-launch period).
I expect my analysis to occur over many weeks/months. I won't be making any great discoveries today but I am going to kick off the process.
Model Selection Criteria
I decided to start by analyzing third party R2G stock models with at least 180 days of post-launch performance. In addition, I decided to only look at models that hold 20 stocks initially. I expect less volatility and hence more consistency with 20 stock models than those with fewer stocks. I also chose to ignore models that hold more than 20 stocks.
Pre- and Post-Launch Time Periods
My first step was to download the daily equity data for eleven models that meet my criteria into a spreadsheet. I then separated the data into two time periods. The first time period was from April 6, 2004 to Dec 31, 2012. This is the pre-launch time period that all of the 11 models have backtest performance data for.
The second time period is Aug 7, 2013 to Feb 7, 2014. This is the last six months of post-launch. All 11 models have been "live" for this time period.
Input(s) Versus Outputs(s)
I don't particularly like predicting annual excess returns as this output doesn't tell the whole story regarding drawdown, alpha, beta, etc. I have decided to initially look at excess Sortino Ratio. By "excess" I mean compared to a benchmark as opposed to risk-free return.
Since I will be looking at post-launch excess Sortino Ratio, I felt it would be logical to also look at pre-launch excess Sortino Ratio as a possible predicter.
As a first attempt, I chose the SPDR S&P 500 ETF trust as the benchmark, although many of the models are actually smallcap or microcap models.
The results are provided on the chart below.
Just to be certain about the choice of benchmark, I decided to create the same chart except with the Russell 2000 ETF used as the benchmark. The results are similar except that the points are shifted to the left on the graph. This suggests that it was more difficult to outperform the Russell 2000 than it was the S&P 500 Pre-Launch, but Post-Launch they were about as equally difficult.
From these charts it can be seen that two models are performing better post-launch than pre-launch: Keating's med turnover and Quantonomics smallcap gems. There are two models that are underperforming: Sherman's with low max drawdown and Thank Buffet.
Above-Average S&P500 is performing in line with the benchmark, indicating that there has been no advantage to trading this model in the last 6 months. The same holds true for Marco's Merger&Acquisition model. And here is where it becomes evident that excess Sortino Ratio is not sufficient. The M&A model is very low beta... it is not expected to outperform the benchmark, but it is expected to have lower drawdown, and that is why looking at excess returns or any kind of excess performance does not tell the whole story.