2
$\begingroup$

I would like to model the Value-at-Risk of U.S. sector indices and the U.S. Broad Dollar Index using the variance-covariance method. To achieve this, I model the conditional means and variances of the returns using ARMA-GARCH models. Here is the issue: I first need to determine whether ARMA orders are necessary to eliminate whatever autocorrelation may be present in the returns series. Convention indicates that Ljung-Box tests are in order, however, having researched the topic more, I came across the automatic Portmanteau test for serial correlation as seen here, which supposedly addresses some of the shortcomings of the Ljung-Box test, namely the issues of the selection of a superficial lag order, low power, and lack of robustness to heteroskedasticity. The equivalent R package is vrtest, and more specifically, the Auto.Q command.

My anxieties with this statistic lie in the fact that I get vastly different results using it as opposed to the Ljung-Box test through the Box.test command. This is true almost across the board with my sector indices, but for instance, the CRSP Real Estate Index yields the following results:

> Auto.Q(ts_realestate_is, lags=20)
$Stat
[1] 1.159207

$Pvalue
[1] 0.28163

> Box.test(ts_realestate_is, lag = 10, type = "Ljung")

    Box-Ljung test

data:  ts_realestate_is
X-squared = 187.59, df = 10, p-value < 2.2e-16

If you follow the results of the Ljung-Box test, then you would come to the conclusion that there is likely serious autocorrelation you need to address with possibly some ARMA model before applying a GARCH-type model, but the Automatic Portmanteau test indicates that the returns series is likely some sort of white noise, or at least that there might not be a reason to apply an ARMA model to address serial correlation. This naturally raises a question: should I trust the automatic portmanteau test over the Ljung-box despite these large disparities? I would greatly appreciate any help on this matter.

If it is of relevance, after detecting autocorrelation through the Automatic test, I used auto.afirma to determine optimal ARMA orders by minimizing AIC, and then tested the standardised residuals again with the Automatic test to ensure that no autocorrelation remains. When done exclusively through the Ljung-Box test, I almost always found that none of the optimal models of auto.arfima had white noise for the residuals as evaluated by a 10-lag Ljung-Box test. This was another reason why I wanted to use another method; it did not make sense for there to be significant remaining autocorrelation for the residuals of such returns series after fitting ARMA models.

Yes, I do understand that ARMA-GARCH orders should ideally be determined in parallel, but I do not currently have a good way of doing so. This is why I am opting to address autocorrelation separately. However, from what I understand from the authors of the test, it should also work for the residuals of some ARMA-GARCH model as well.

$\endgroup$
1
  • $\begingroup$ What do you think about my answer? If it is helpful and clear, you may accept it by clicking on the tick mark to the left. Otherwise, you may ask for further clarification. A helpful answer can also be upvoted by clicking on the upward-pointing arrow. This is how Cross Validated works. $\endgroup$ Jun 4, 2023 at 12:47

1 Answer 1

1
$\begingroup$

Some points:

  1. The results of Auto.Q and Box.test may differ not only due to the peculiarities of the test statistics, but also because of the different lag lengths that are used. For a direct comparison between the test results, specify the same lag length in Box.test as you get in Auto.Q.

  2. Ljung-Box test might not be appropriate for diagnostic testing of ARMA models; see "Testing for autocorrelation: Ljung-Box versus Breusch-Godfrey". I have no idea whether the same applies to the Automatic Portmanteau test as well.

  3. When choosing an ARMA model by AIC, you aim for a model that is best at prediction. That need not be the model that has white noise residuals. The additional complexity of the model needed to make the residuals white noise may not be justified from the forecasting perspective. This is about the bias-variance trade-off. AIC-based trade-off will be different than obtained by seeking white noise residuals.

$\endgroup$
6
  • $\begingroup$ 1: I cannot specify exactly which lag to use, as one of the main features is the automatic selection of lags. Across a wide range of lags, I see that the Ljung-Box Q only ever increases, and at no point reaches anything near what Auto.Q outputs. 2: I cannot use Breusch-Godfrey because the R command doesn't support ARMA(P,Q) models. I opted for Auto.Q as a compromise for the inconsistencies of Ljung. 3: I understand that AIC has a different goal. Would it not be fair to argue that white noise is needed to ensure that the ARMA-GARCH models hold in terms of assumptions? $\endgroup$
    – WebSurfer
    May 4, 2023 at 10:08
  • $\begingroup$ @WebSurfer, 1: You can specify the same lag in Ljung-Box as in Auto.Q, since for Ljung-Box the lag is chosen by the user rather than automatically. 2: Are there indications that make Auto.Q immune to the problem of Ljung-Box in the context of ARMA residual diagnostics? 3: Yes. If you care about the statistical adequacy of the model more than about its predictive ability, you may give up AIC for something else. $\endgroup$ May 4, 2023 at 10:32
  • $\begingroup$ 1: I may not be understanding the command correctly, but it doesn't appear to show me the chosen lag, so I cannot match them. 2: I do not claim to understand the authors' paper very well, but as I understand, they claim that their test is suitable for dynamic models, as opposed to Ljung-Box, at least conceptually. 3: I think for VaR modelling, a mix of both may be necessary, as in a high-AIC-ranking model with sufficiently uncorrelated errors. Though, I understand the necessity of compromise here. Still, I do not quite understand the discrepancy between these statistics. $\endgroup$
    – WebSurfer
    May 4, 2023 at 13:10
  • $\begingroup$ @WebSurfer, 1: Oh, I see. Well, that is unfortunate (unless there actually is a way to extract the lag order from the output). 2: Sounds reasonable. 3: Sounds reasonable, though why would you be puzzled by the discrepancy? Thinking about all possible models, there is a curve in terms of bias-variance trade-off. AIC-optimal model occupies one point on it while a model selected by seeking white noise residuals occupies another point. There is no reason for them to coincide. $\endgroup$ May 4, 2023 at 13:14
  • $\begingroup$ Sorry, in my last statement, I meant that I'm still rather unsure about why there is such a large difference between my Ljung-Box Q statistic and this Automatic Q statistic, not between AIC-chosen and no-autocorrelation models. $\endgroup$
    – WebSurfer
    May 4, 2023 at 13:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.