Comparing ARIMA and VAR
I want to evaluate the performance of VAR (Vector Autoregression) and ARIMA (Auto regressive Integration Moving Average) using two time series X and Y. I have used R. So far, it seems ARIMA (using only Y and excluding X) is performing better than VAR (using both X and Y) but I am afraid I am doing something wrong. Let me explain.
First, my main goal is to understand if VAR is a good model to predict values of a time series Y using both the historical values of Y and another time series X. I assume that X and Y are endogenous time series. The foundations of VAR made more sense to me than using ARIMA with “exogenous regressors” (using Y for the regressors) due to dependency assumptions.
I have tried to follow different statistical tests for VAR before making any prediction, these tests are the following:
- Test if the time series need to be log transformed, if they do, I log transform them
- I normalize the time series using z-transformation so both are in the same scale
- I hold out 1 month (the last one) for testing and keep the rest for training
- I test if they need to be differentiated and if there is seasonality. Using first the R function nsdiffs, I fix the seasonality lag in case that it is bigger than 0. Then, using the R function ndiffs I fix the integration lag (using Augmented Dickey–Fuller test) in case that it is bigger than 0 (some times X has a lag >0 and Y does not have a lag > 0).
- I reduce the time window for both time series as to eliminate NA values (NA emerge when you differentiate for seasonality or stationarity) and make sure that both have the same size.
- Select the lag for the VAR making sure they are not auto correlated (using test "PT.asymptotic") using R function VARselect.
- Test for Granger causality and normality
- Make predictions with VAR and evaluate the result. Alternatively, using only the time series Y I use ARIMA, using auto.arima function from R
Well now, here my question comes. The out of sample data is only one month. To check if the prediction power last in time, I go back in time one month at a time and train again my whole model for VAR and ARIMA. For example, if my training time series X and Y had 87 observations I initially predict the data point 88 using 87 observations for training. Next, I use the 86 first observations and predict the 87th data point. To do that I train a new model using the 86 observation both for VAR and ARIMA. I repeat this process until I have a minimum number of 36 observations (3 years) and I only consider cases where the training set passes all the statistical tests for VAR. When I do this, I obtain much better results with ARIMA specially after I go back in time for 2 or 3 months. I wonder if I am doing something wrong. Why all of a sudden the VAR model gets so unstable?.VAR is supposed to perform better than ARIMA, no?
Please see the results below, the time series plotted is Y and the red points are the values predicted as I go back in time. The black thin line links the predictions. At the beginning VAR performs better but after the 2nd month back in time it performs terribly.