1
$\begingroup$

I estimated a GARCH model to forecast the variance of a variable conditional on past information. I evaluated the forecast by comparing the squared forecast error, i.e. the squared value by which the conditional mean model forecast missed the actual value, with the forecasted conditional variance.

It turned out to be a terrible forecast on average (I did this for a multitude of time-series, i.e. I am working in a panel setting). Specifically, a small number of extremely bad forecasts in terms of the aforementioned evaluation appear to "make it bad" on average.

I then used the absolute values of the residuals as my dependent GARCH model variable instead of squared values, as the classic GARCH approach suggests. I compared the forecast values resulting from this model with the absolute value by which the conditional mean model forecast missed the actual value. It turned out to be much better on average.

Now my question is: Is it generally fine to use absolute values of the residuals instead of squared values in a GARCH model? From my understanding, instead of modelling the conditional variance one would then simply model the conditional standard deviation. Is this correct? Thank you in advance.

$\endgroup$
6
  • $\begingroup$ Are you aware that squared residuals are noisy proxies of the underlying conditional variance and that under the true model you would get an $R^2$ of somewhere around 0.3 (not 1.0)? Also, when you say you did the same thing, do you mean the model remained the same but you looked at absolute values of errors rather than squared errors? Or did you fit a model for $\sigma_t$ rather than $\sigma_t^2$? In the first case, you do not gain anything* as the model is still the same, just your evaluation measure is on a different scale so it produces values that may subjectively look better. $\endgroup$ Feb 22 at 14:36
  • $\begingroup$ *Modeling conditional variance vs. modeling conditional standard deviation is exactly the same thing as they are deterministically related and there is a bijection between the two. You change the model's representation but not its content. $\endgroup$ Feb 22 at 14:37
  • $\begingroup$ I am aware of the fact that they are a very noisy proxy, yes. By saying I did the same thing I meant the latter case you describe, i.e. my model's dependent variable were the absolute values of the residuals, not the squared ones. I actually just found a GARCH glossary by Bollerslev (public.econ.duke.edu/~boller/Papers/glossary_arch.pdf). On page 30 (TS-GARCH), he describes exactly the case I describe. He also states that such a model is not as vulnerable to extreme values, thus providing evidence for the fact that my conditional SD model did better than the conditional variance model. $\endgroup$
    – shenflow
    Feb 22 at 15:47
  • $\begingroup$ I also edited the question slightly. Hopefully, it becomes clearer now. $\endgroup$
    – shenflow
    Feb 22 at 15:49
  • $\begingroup$ The dependent variable of your model is most likely a (logarithmic) financial return or an excess return. It is neither its absolute value nor its square. The "scale" equation of GARCH usually has $\sigma_t^2$ on the left hand side, but you chose to have $\sigma_t$ instead. This way you did change the model's content, not only representation; this is the second case of what I mentioned above. This is fine. You just have to be careful comparing performance of the models; make it apples to apples, not apples to oranges. Use out of sample absolute values in both cases or squares in both cases. $\endgroup$ Feb 22 at 16:34
1
$\begingroup$

A standard GARCH model would be something like \begin{aligned} x_t &= \mu_t+\varepsilon_t, \\ \varepsilon_t &= \sigma_t z_t, \\ \sigma_t^2 &= \omega+\alpha_1\varepsilon_{t-1}^2+\beta_1\sigma_{t-1}^2, \\ z_t &\sim i.i.d.(0,1) \end{aligned} for some conditional mean $\mu_t$, e.g. $\mu_t=\mu$ or $\mu_t=\varphi_1 x_{t-1}$.

You substitute $\sigma_t^2=\omega+\alpha_1\varepsilon_{t-1}^2+\beta_1\sigma_{t-1}^2$ with $\sigma_t=\omega+\alpha_1|\varepsilon_{t-1}|+\beta_1\sigma_{t-1}$, and this is fine.

Regardless of which model you use, when you are evaluating it, you will likely get a better fit from regressing the fitted (or predicted) $\hat\sigma_t$ on the absolute valus of residuals (or prediction errors) $|\hat\varepsilon_t|$ than from regressing $\hat\sigma_t^2$ on $\hat\varepsilon_t^2$.* If you are going to compare different models, make sure to compare fit in an apples-to-apples way. What you should not do is compare the fit of $\hat\sigma_t$ on $|\hat\varepsilon_t|$ from one model with the fit of $\hat\sigma_t^2$ on $\hat\varepsilon_t^2$ from another model; that would be an apples-to-oranges type of comparison.

*There are also other ways of evaluating the fit of the models. Proper scoring rules is a relevant keyword.

$\endgroup$
2
  • $\begingroup$ I do not necessarily want to compare the "fit" but rather the forecast performance. But probably this is what you meant by "fit", right? Also, help me understand how this is an apples-to-oranges comparison: Is it due to the fact that the in one case, I am forecasting the conditional variance and in the other case, I am forecasting the conditional standard deviation? If this is what you mean - I got it. Quite trivially, any error in terms of forecasted conditional SD will translate into its squared value in terms of forecasted conditional variance. $\endgroup$
    – shenflow
    Feb 22 at 17:25
  • $\begingroup$ @shenflow, you got me right. Also, when I said fit, I meant $R^2$ of regresssion or any analogous measure. You can run the regression on in-sample fitted values or, since you are interested in forecasts, on out-of-sample forecasts. In either case the regression will fit better or worse e.g. as measured by $R^2$. $\endgroup$ Feb 22 at 17:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.