I have generated two VAR time series models in R for a dataset. My query is how can I compare those two models based on any kind of metrics like forecasting power :accuracy / f-1 score or something which can tell that 1st model is say 70% better than the second model.
I need some comparison metrics code in R. Thanks in advance
1. Likelihood ratio test / Vuong's test
If the two models are nested, you could do a likelihood ratio test. If they are not nested, Vuong's test could be relevant.
2. AIC
You could compare the models' AIC values. Just make sure the dependent variables are exactly the same and the observations on which the model likelihood is based are the same. (This would usually not be the case if you naively compare VAR(p) with VAR(q) for $p\neq q$. Cut $|p-q|$ initial observations from the model with the lower autoregressive order to fix that. Having different sets of dependent variables such as a bivariate VAR vs. a trivariate VAR would also be problematic.)
3. Out-of-sample forecast losses
You could compare the two models' out-of-sample forecast errors and the corresponding forecast losses (functions of forecast errors) via time-series cross validation. You could look at, say, mean absolute error or mean squared error or whatever loss function is relevant for you.
4. Diebold-Mariano test
You could use the Diebold-Mariano test (see diebold-mariano), though it is primarily intended for forecast comparison rather than model comparison.
The different methods of comparison answer somewhat different questions. You may choose one of them depending on what your goal is.
