# Evaluating results for VAR and ARIMA

Comparing ARIMA and VAR

Hi everybody,

I want to evaluate the performance of VAR (Vector Autoregression) and ARIMA (Auto regressive Integration Moving Average) using two time series X and Y. I have used R. So far, it seems ARIMA (using only Y and excluding X) is performing better than VAR (using both X and Y) but I am afraid I am doing something wrong. Let me explain.

First, my main goal is to understand if VAR is a good model to predict values of a time series Y using both the historical values of Y and another time series X. I assume that X and Y are endogenous time series. The foundations of VAR made more sense to me than using ARIMA with “exogenous regressors” (using Y for the regressors) due to dependency assumptions.

I have tried to follow different statistical tests for VAR before making any prediction, these tests are the following:

1. Test if the time series need to be log transformed, if they do, I log transform them
2. I normalize the time series using z-transformation so both are in the same scale
3. I hold out 1 month (the last one) for testing and keep the rest for training
4. I test if they need to be differentiated and if there is seasonality. Using first the R function nsdiffs, I fix the seasonality lag in case that it is bigger than 0. Then, using the R function ndiffs I fix the integration lag (using Augmented Dickey–Fuller test) in case that it is bigger than 0 (some times X has a lag >0 and Y does not have a lag > 0).
5. I reduce the time window for both time series as to eliminate NA values (NA emerge when you differentiate for seasonality or stationarity) and make sure that both have the same size.
6. Select the lag for the VAR making sure they are not auto correlated (using test "PT.asymptotic") using R function VARselect.
7. Test for Granger causality and normality
8. Make predictions with VAR and evaluate the result. Alternatively, using only the time series Y I use ARIMA, using auto.arima function from R

Well now, here my question comes. The out of sample data is only one month. To check if the prediction power last in time, I go back in time one month at a time and train again my whole model for VAR and ARIMA. For example, if my training time series X and Y had 87 observations I initially predict the data point 88 using 87 observations for training. Next, I use the 86 first observations and predict the 87th data point. To do that I train a new model using the 86 observation both for VAR and ARIMA. I repeat this process until I have a minimum number of 36 observations (3 years) and I only consider cases where the training set passes all the statistical tests for VAR. When I do this, I obtain much better results with ARIMA specially after I go back in time for 2 or 3 months. I wonder if I am doing something wrong. Why all of a sudden the VAR model gets so unstable?.VAR is supposed to perform better than ARIMA, no?

Please see the results below, the time series plotted is Y and the red points are the values predicted as I go back in time. The black thin line links the predictions. At the beginning VAR performs better but after the 2nd month back in time it performs terribly.

• The pattern in the VAR forecasts seem to suggest that they're being generated by two different models. For example, the 2014 VAR forecasts are probably from a model in differences (since they hover around zero) while the 2015 VAR forecasts are probably from a model in levels (since they don't return to a mean of close to zero). Can you confirm whether or not this is the case? – Graeme Walsh Sep 29 '16 at 21:52
• @GraemeWalsh thanks. The first 3 predicted points come from a model with a – ruthy_gg Sep 29 '16 at 22:16
• @GraemeWalsh thanks. Lets call the time series in question Y (unemployment) and the other time series as X. All the predicted values have a VAR(3) except the last one (the isolated one in 2011) with a VAR(4). The first 3 predicted points come from a model with difference value of 1 for the X time series. The 4th-7th predicted values come from a model with difference of 2 for the Y time series and 1 for the X time series. The 8th-14th predicted value come from a model with difference of 2 for X and Y. No seasonality detected. Should I keep the starting model for all the predictions? – ruthy_gg Sep 29 '16 at 22:26
• Would this be allowed given that I will fit a model based on a model built from future values? – ruthy_gg Sep 29 '16 at 22:27
• My hunch tells me that the changing order of integration (number of differences) is the crux of your problem. Maybe it'd be better to simplify your method by using the same model for estimation and then generate two types of forecasts (i) a sequence of one-period ahead forecasts, and (ii) an n-step ahead set of forecasts? For the model, you can choose to re-estimate the parameters or not - depending on what the purpose of your research is. If you do use a mix of differences, just make sure to undifference the forecasts where appropriate. Does it make sense? – Graeme Walsh Sep 29 '16 at 22:39