Validity of Automatic Portmanteau test for serial correlation vs Ljung-Box Test

Question

I would like to model the Value-at-Risk of U.S. sector indices and the U.S. Broad Dollar Index using the variance-covariance method. To achieve this, I model the conditional means and variances of the returns using ARMA-GARCH models. Here is the issue: I first need to determine whether ARMA orders are necessary to eliminate whatever autocorrelation may be present in the returns series. Convention indicates that Ljung-Box tests are in order, however, having researched the topic more, I came across the automatic Portmanteau test for serial correlation as seen here, which supposedly addresses some of the shortcomings of the Ljung-Box test, namely the issues of the selection of a superficial lag order, low power, and lack of robustness to heteroskedasticity. The equivalent R package is vrtest, and more specifically, the Auto.Q command.

My anxieties with this statistic lie in the fact that I get vastly different results using it as opposed to the Ljung-Box test through the Box.test command. This is true almost across the board with my sector indices, but for instance, the CRSP Real Estate Index yields the following results:

> Auto.Q(ts_realestate_is, lags=20)
$Stat
[1] 1.159207

$Pvalue
[1] 0.28163

> Box.test(ts_realestate_is, lag = 10, type = "Ljung")

    Box-Ljung test

data:  ts_realestate_is
X-squared = 187.59, df = 10, p-value < 2.2e-16

If you follow the results of the Ljung-Box test, then you would come to the conclusion that there is likely serious autocorrelation you need to address with possibly some ARMA model before applying a GARCH-type model, but the Automatic Portmanteau test indicates that the returns series is likely some sort of white noise, or at least that there might not be a reason to apply an ARMA model to address serial correlation. This naturally raises a question: should I trust the automatic portmanteau test over the Ljung-box despite these large disparities? I would greatly appreciate any help on this matter.

If it is of relevance, after detecting autocorrelation through the Automatic test, I used auto.afirma to determine optimal ARMA orders by minimizing AIC, and then tested the standardised residuals again with the Automatic test to ensure that no autocorrelation remains. When done exclusively through the Ljung-Box test, I almost always found that none of the optimal models of auto.arfima had white noise for the residuals as evaluated by a 10-lag Ljung-Box test. This was another reason why I wanted to use another method; it did not make sense for there to be significant remaining autocorrelation for the residuals of such returns series after fitting ARMA models.

Yes, I do understand that ARMA-GARCH orders should ideally be determined in parallel, but I do not currently have a good way of doing so. This is why I am opting to address autocorrelation separately. However, from what I understand from the authors of the test, it should also work for the residuals of some ARMA-GARCH model as well.

Answer 1

Some points:

1. The results of Auto.Q and Box.test may differ not only due to the peculiarities of the test statistics, but also because of the different lag lengths that are used. For a direct comparison between the test results, specify the same lag length in Box.test as you get in Auto.Q.

2. Ljung-Box test might not be appropriate for diagnostic testing of ARMA models; see "Testing for autocorrelation: Ljung-Box versus Breusch-Godfrey". I have no idea whether the same applies to the Automatic Portmanteau test as well.

3. When choosing an ARMA model by AIC, you aim for a model that is best at prediction. That need not be the model that has white noise residuals. The additional complexity of the model needed to make the residuals white noise may not be justified from the forecasting perspective. This is about the bias-variance trade-off. AIC-based trade-off will be different than obtained by seeking white noise residuals.