1
$\begingroup$

I want to compare the prediction capacity of multivariate models (MGARCH, VAR, SVAR, VECM...). One option to do this is by comparing its error metrics: the mean square error or any other between the predicted values ​​and some test data.

However, I would also like to compare its predictive ability with hypothesis tests. In the univariate case there is the Diebold-Mariano test. There is even a multivariate version of this test, but it compares several predictions but not multivariate models.

I would like to know if you know of another test that allows this comparison between multivariate models. Or another method that allows to vectorize the prediction errors to compare the models.

$\endgroup$
1

1 Answer 1

0
$\begingroup$

If you want to rank the models (and then test whether the best model beats the other models), you will have to define a scalar performance metric (a scalar-valued loss function). With that in place, and since you have more than two models to compare, I refer you to the tag description of :

In comparing multiple forecasts, we need to address the multiple comparisons problem. In such a case, the standard approach is the "multiple comparisons to the best" (MCB) test originally proposed by Koning et al. (2005). <...> It is rank-based, so it works with any accuracy measure. <...> A related alternative would be the Friedman-Nemenyi test (Demsar, 2006). Both the MCB and the Nememyi test are implemented in the TStools package for R.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.