August 20, 2014
In a new working paper, UCLA’s Roger Farmer responds to last year’s investigation into his claim that declines in the stock market caused the Great Recession. Farmer apparently failed to grasp the nature of the critique.
In his original paper, Farmer claimed to have found a stable relationship between the movements in S&P 500 and unemployment rates, and that the data “leads me to stress asset market intervention as a potential policy resolution to the problem of high and persistent unemployment.” In other words, the government should deliberately prop up the stock market as a way of boosting the economy. Farmer appealed to the apparent forecasting power of his model to support his policy preference.
In response we countered that his visual evidence of forecasting power was deceptive—playing off the serial correlation in the data to trick the naïve observer. Rather, his model was not in fact powerful, as was demonstrated by the fact that a simpler model that ignored stock prices produced superior forecasts. Our analysis showed that even if Farmer’s model was correct, movements in the stock market fail to explain—let alone cause— the Great Recession. Finally, we pointed out that the intervention necessary to prevent the recession was implausibly large to be considered serious.
Farmer now
- Asserts as fact certain properties of his data shown to be consistent with, but unsupported by his analysis.
- Argues that the asserted properties require the use of a particular type of model.
- Suggests that despite using the proper kind of model, his model was “seriously mispecified” by failing to account for a structural break.
- Reasserts that despite this structural break the observed relationship is somehow “structurally stable.”
- Abandons the “correct way to model” and employs pre-break data in an effort to support the uncontroversial position that stock market data may help forecast unemployment.
We agree that his model may have failed due to structural breaks. In fact, post-2008 data may be completely different in structure than data prior, and therefore any model based on previous data is liable to produce forecasts only spuriously related to the post-2008 economy. In any case, we believe this undermines both his assertion that stock prices caused the Great Recession and his proposed policy solution.
Economists have long accepted causal links between the stock market and the real economy. There may, for example, be “wealth effects” associated with movements in stock prices. As stock prices rise, stockholders may feel wealthier and demand additional goods and services. But causation may run the other direction. Stock prices are generally accepted to reflect information regarding future profits and therefore serve as leading indicators of the future economy. (Then again, bubbles happen, so they are certainly imperfect indicators.) News that indicates lower employment and profits tomorrow may lower stock prices today. Furthermore, stockholders may sell their holdings if they lose their jobs—especially if unemployment spells are lengthy– possibly making stock prices a lagging indicator.
These links may reinforce one another in vicious cycles, but Farmer’s work here does not obviously add to these understandings.
In any case, Farmer offers his new model in reply to the fact that the “univariate” model of our first response (the simple model which ignores movements in the stock market) produced better out-of-sample forecasts of unemployment than did Farmer’s original model. Instead, Farmer missed the point of presenting the simple model. I had never expected the univariate model to outperform Farmer’s original. Our point was that it is very easy to get a seemingly good fit—not that the market provides misleading information and ought to be left out of any model. Apparently, the fact that the univariate model did produce better forecasts obscured our point, rather than augmented it.
Regardless, Farmer makes a series of contradictory and unsubstantiated claims. He reasserts that he “showed that the [original] model provides an excellent fit to data from 1979q4 through 2011q1.“ Again, the entire point of the univariate model in our initial response was to show just how easy it is to get an “excellent” fit to the data. The fact that Farmer can propose theories as to why his original model performed poorly out of sample does not support his case.
Farmer also claims his data comprise “non-stationary cointegrated series.” However, his tests are very weak and employ the null hypothesis that they contain a unit root. Farmer was unable to reject that hypothesis—which is to say he did not find evidence that his data series are stationary. This hardly means they are not stationary. However, if they are both non-stationary and cointegrated (and he does present evidence that if they are non-stationary that they are cointegrated) then in not accounting for cointegration his new model produces misleading results.
To be clear, we do not object to Farmer’s proceeding on the assumption that the series are non-stationary. We are concerned that he first claims it as an established fact, then states “The correct way to model a pair of non-stationary cointegrated time series is with a VECM”[1] and he thereafter proceeds to ignore those previous claims by abandoning the VECM approach. His justification appears to be that there may be a structural break in the data that hurts model performance.
But if the original model is to be abandoned on the grounds that the post-1979 data is statistically distinct, then why base the new model on pre-1980 data? Where exactly does this structural break take place? Farmer asserts that it makes sense to place it between the third and fourth quarters of 1979. To examine this possibility, we examine 25-year windows of data and report the estimated cointegration coefficient on stock prices. Figure 1 shows those estimated coefficients as the window of data shifts.
It does appear that there is indeed some drift in the long-run relationship between the stock market and unemployment in the early 1990s (as the data become more post-1979 than pre-1980.) However, the relationship really changes as soon as the mid-1980s data exits and the late 2000s data comes into play. It is not clear that there is a cointegrating relationship over the 1986-2011 period. By varying the size of the window, we see that it is the inclusion of post-2008 data, rather than the exclusion of the late 1980s data that matters. We will return to this bit of information later.
Farmer shows that his new model (based on data through 1979) does a poor job of forecasting the recession in late 2007. He defends this result, saying, “That is unsurprising since, at this date, the stock market had not yet begun its spectacular decline.” He proceeds to show how a forecast of unemployment made in late 2008 performs much better.
However, delaying the forecast date is not a good way to incorporate the as-of-yet unrevealed information on future stock prices. Farmer’s forecast depends on the assumption that there are no unexpected movements in stock prices. (This is why we offered Figure 5 in out initial reply.) That is, the high unemployment seen in his Figure 6 depends on the stock market remaining depressed all through 2010. Farmer’s forecast should have overshot significantly on unemployment having not anticipated the rapid gains in the stock market over 2009 and 2010.
This last point ties back into our earlier observation that the post-2008 data appears to be structurally different from pre-2009. The sudden improvement in the model’s power to predict unemployment is less likely to be mere inclusion of new information than it is a fluke. Still, if we do accept the model as valid, we may observe that anticipation of the recovery in the stock market recovery helps foresee a modestly lower rate of unemployment without accepting the existence of especially large wealth effects.
In fact, Farmer backs away from claiming causality in the ordinary sense, writing “In my view, that drop [in housing wealth] triggered a subsequent increase in the unemployment rate. But it was the precipitous crash in the stock market, beginning in the fall of 2008, that turned an otherwise mild contraction into what we now refer to as the Great Recession.”
That is one view, but not one especially supported by Farmer’s papers. Perhaps traders were simply overconfident in the early stages of the Great Recession. One might as easily write, “The 2008 financial crisis helped reveal an otherwise mild contraction to be what we now refer to as the Great Recession, causing additional declines in the stock market.”
Farmer’s papers leave unanswered this critical question: to what extent does future output fall because the stock market declined today as opposed to how much the stock market falls today because of an anticipated fall in output? Respecting both effects, the stock market today yields information that may be used to predict future output. The market may thus “Granger-cause” unemployment, but this tells us nothing about the underlying causes of the Great Recession and therefore does not point us to asset market intervention as a policy solution.
[1] VECM stands for “vector error-correcting model.” VECMs account for cointegration between data series, thus helping to prevent spurious relationships.
