Why is GARCH offering no predictive value?

Question

I am playing around with GARCH models for the first time (I have a stats background but basically no experience with GARCH), trying to forecast volatility in a financial time series.

I trained a GARCH(1,1) model on 3,000 data points and forecasted 1 period ahead 500 times (retraining to include new data point after each prediction is made). Below are my results (the points/circles is the original time series, the line is the GARCH volatility prediction for that timepoint).

Please correct me if I'm wrong, but it seems that the GARCH model offers no predictive value. The "predicted" volatility values spike up after a big price move. I feel like these results could be replicated by just taking a rolling window of realized past volatility (as the GARCH model appears lagging, not predictive).

Does this seem wrong, or is this what results usually look like? Has anyone had any success with something like this? Any advice for what I might be doing wrong?

I have tried specifying the mean model as ARMA(0,0) and ARMA(1,1), no significant difference.

EDIT: I am adding my code to better supplement my question. To be clear, my data is NOT time bars, but volume bars (price sampled every time a certain threshold of volume is traded) as they have been shown to have better statistical properties. These volume bars are sampled, on average, every 4 hours (though changes significantly based on the level of trading activity).

library(rugarch)
library(forecast)
volumebardata <- read.csv(file='MyCSVFile', header=TRUE)

returns <- function(vector){
  rets <- c()
  for (i in 2:length(vector)){
    rets <- c(rets, vector[i]/vector[i-1])
  }
  return(rets)
}

pricereturns <- (returns(volumebardata$VolumeBarClose))-1
priceretstrain <- pricereturns[1:3000]
priceretstest <- pricereturns[3001:3500]

# Specify a standard GARCH model with constant mean
garchspec <- ugarchspec(mean.model = list(armaOrder = c(0,0)),
                        variance.model = list(model = "eGARCH", 
                                              garchOrder=c(1,1),
                                              variance.targeting=FALSE), 
                        distribution.model = "std")


# Estimate the model
garchfit <- ugarchfit(data = priceretstrain, spec = garchspec)

predvol <- c()
for (i in 1:300){
  if (i > 1){
    # Specify a standard GARCH model with constant mean
    garchspec <- ugarchspec(mean.model = list(armaOrder = c(0,0)),
                            variance.model = list(model = "eGARCH", 
                                                  garchOrder=c(1,1),
                                              variance.targeting=FALSE), 
                            distribution.model = "std")
    
    # Estimate the model
    fulldata <- c(priceretstrain, priceretstest[1:(i-1)])
    garchfit <- ugarchfit(data = fulldata, spec = garchspec)
  }
  # Forecast volatility 1 period ahead  
  garchforecast <- ugarchforecast(fitORspec = garchfit, 
                                  n.ahead = 1)
  # Extract the predicted volatilities and print them
  predvol <- c(predvol, sigma(garchforecast))
}
plot(cumprod(priceretstest+1)[1:500], type='l')
par(new=TRUE)
plot(predvol[1:500], type='l')

Additionally, here is a zoomed in plot of the GARCH-predicted volatility (red line) vs. the squared returns (as proxy for "true" volatility, shown in black). You can quite clearly see the time-lag.

EDIT 2: Several commenters are pointing out that I might be measuring volatility incorrectly, therefore of course the GARCH predictions appear incorrect. However, I don't understand why the model is being defended when clearly the volatility predictions lag behind the actual rapid shifts in the time series (regardless of how you measure it), making the predictions obviously useless because I can derive the same kind of predictions with a V(t+1)=V(t) "prediction" model. Am I misusing GARCH? Or is it just not all that great of a volatility prediction model?

Answer 1

First of all, your results look a bit strange. I would advise you to check your code. Nevertheless, I will describe a method that you can use to obtain one-step-ahead forecasts for the conditional variance using a GARCH(1,1)-model.

Method

Assume that you observe a time series $(r_t)_{t=1}^T$ of log-returns and you want to estimate a simple GARCH(1,1) model. \begin{align} r_t&=\sigma_t u_t \quad, u_t \sim \mathcal N(0,1) \\ \sigma_t^2&=\alpha_0+\alpha_1r_{t-1}^2+\beta_1 \sigma_{t-1}^2 \end{align} First of all, estimate the model on the first $N$ observations where $N <T$ and denote the ML estimate as $\hat{\boldsymbol{\theta}}^{j=1}=(\hat{\alpha}_0^{j=1},\hat{\alpha}_1^{j=1},\hat{\beta}_0^{j=1})^\top$. Then calculate the time series $(\sigma_t^2)_{t=1}^N$ as follows:

1. choose an initial estimate for $\sigma_1^2$, for instance $\sigma_1^2=\frac{1}{N}\sum_{t=1}^Nr_t^2$.
2. $\sigma_2^2=\hat{\alpha}_0^{j=1}+\hat{\alpha}_1^{j=1}r_1^2+\hat{\beta}_0^{j=1}\sigma_1^2$
3. $\vdots$
4. $\sigma_N^2=\hat{\alpha}_0^{j=1}+\hat{\alpha}_1^{j=1}r_{N-1}^2+\hat{\beta}_0^{j=1}\sigma_{N-1}^2$

Now, you can predict the conditional variance for $t=N+1$ as $$ \hat{\sigma}_{N+1}^2=E(\sigma_{N+1}^2\vert \mathcal F_{N})=\hat{\alpha}_0^{j=1}+\hat{\alpha}_1^{j=1}r_{N}^2+\hat{\beta}_0^{j=1}\sigma_{N}^2 $$ , which is the MSE optimal prediction. If you want to use a rolling window, re-estimate the model on $(r_t)_{t=2}^{N+1}$ and obtain $\hat{\boldsymbol{\theta}}^{j=2}=(\hat{\alpha}_0^{j=2},\hat{\alpha}_1^{j=2},\hat{\beta}_0^{j=2})^\top$. You can calculate $(\sigma_t^2)_{t=2}^{N+1}$ as described above.

Then, predict $$ \hat{\sigma}_{N+2}^2=E(\sigma_{N+2}^2\vert \mathcal F_{N})=\hat{\alpha}_0^{j=2}+\hat{\alpha}_1^{j=2}r_{N+1}^2+\hat{\beta}_0^{j=2}\sigma_{N+1}^2 $$ You repeat this process until no observations are left. As a result, you have a time series $(\hat{\sigma}_t^2)_{t={N+1}}^T$ which are the predictions of $\sigma_t^2$ using a rolling window.

Evaluation of volatility forecasts

There was a great discussion in the literature, whether GARCH-models are able to provide precise volatility forecasts or not. It turned out that it was not the models that gave bad results, rather many people used "wrong" proxies for volatility. (Reference: : Torben G. Andersen and Tim Bollerslev (1998): "Answering the Skeptics: Yes, Standard Volatility Models do Provide Accurate Forecasts", in International Economic Review, Vol. 39, No. 4).

In sum, one of the major problems when evaluating volatility forecasts is that volatility is unobservable and you need to use some form of proxy. Assuming that the specified model is correct, an unbiased estimator of the "true" volatility $\sigma_t^2$ is given by the squared returns $r_t^2$ because: $$ E(r_t^2 \vert \mathcal F_{t-1})=E(\sigma_t^2u_t^2 \vert \mathcal F_{t-1})=\sigma_t^2E(u_t^2)=\sigma_t^2 $$ Thus, you could plot $r_t^2$ and $\hat{\sigma}_t^2$ to assess whether the results make sense to some extent. Usually, a simple GARCH(1,1)-model does a moderate job in predicting $\sigma_{t+1}^2$. Exceptions prove the rule, but if the results are completely different, it is likely that there is an error in the code.

However, note that $r_t^2$ is a noisy proxy for $\sigma_t^2$. Usually, you get much better results, if you don't use $r_t^2$ as a proxy for $\sigma_t^2$ but realized volatility estimators like $$ RV_{t,n}=\sum_{i=1}^n(\ln(P_{t,i})-\ln(P_{t,i-1})). $$ So, it is possible that your code is correct but for your time series, $r_t^2$ is a really bad proxy for the unobservable volatility and you may get completely different results if you use RV. However, to do that, you need to have access to intraday data and getting the data is quite a challenge if you don't have access to Bloomberg or other data providers.

Answer 2

Your observation is correct. GARCH is an autoregressive model and its $h$-step-ahead predictions tend to lag $h$ steps behind, as is the case with most autoregressive models.

We often model time series processes as being hit by a new zero-mean stochastic shock every period. A special case that illustrates the lagging predictions best is an AR(1) with a zero intercept and a unit slope (in other words, a random walk): $$ y_t=c+\varphi_1 y_{t-1}+\varepsilon_t $$ where $c=0$ and $\varphi_1=1$. An optimal (under square loss) $h$-step-ahead point forecast is $\hat y_{t+h|t}=y_t$, i.e. the last observed value. Thus even if we were able to estimate $c$ and $\varphi_1$ with perfect precision, our optimal (!) forecast would seem to lag by $h$ steps.

Similar logic applies in the more general case of $c\neq 0$ and $\varphi\neq 1$, though the argument for the general case is more nuanced. GARCH being an autoregressive model suffers from the same problem. (The fact that GARCH is autoregressive in terms of conditional variance rather than conditional mean does not change the essence. See this answer for more detail.) But recall that that need not be a sign of forecast suboptimality, as even optimal forecasts may be characterized by it. This applies to GARCH to a large extent; in typical applications of GARCH models, conditional variance is often found to be quite close to a random walk.