VAR forecasting methodology

Question

I am building a VAR model to forecast the price of an asset and would like to know whether my method is statistically sound, whether the tests I have included are relevant and if more are needed to ensure a reliable forecast based on my input variables.

Below is my current process to check for Granger causality and forecast the selected VAR model.

require("forecast")
require("vars")

#Read Data
da=read.table("VARdata.txt", header=T)
dac <- c(2,3) # Select variables
x=da[,dac]

plot.ts(x)
summary(x)

#Run Augmented Dickey-Fuller tests to determine stationarity and differences to achieve stationarity.
ndiffs(x[, "VAR1"], alpha = 0.05, test = c("adf"))
ndiffs(x[, "VAR2"], alpha = 0.05, test = c("adf"))

#Difference to achieve stationarity
d.x1 = diff(x[, "VAR1"], differences = 2)
d.x2 = diff(x[, "VAR2"], differences = 2)

dx = cbind(d.x1, d.x2)
plot.ts(dx)

#Lag optimisation
VARselect(dx, lag.max = 10, type = "both")

#Vector autoregression with lags set according to results of lag optimisation. 
var = VAR(dx, p=2)

#Test for serial autocorrelation using the Portmanteau test
#Rerun var model with other suggested lags if H0 can be rejected at 0.05
serial.test(var, lags.pt = 10, type = "PT.asymptotic")

#ARCH test (Autoregressive conditional heteroscedasdicity)
arch.test(var, lags.multi = 10)

summary(var)

#Granger Causality test
#Does x1 granger cause x2?
grangertest(d.x2 ~ d.x1, order = 2)

#Does x2 granger cause x1?
grangertest(d.x1 ~ d.x2, order = 2)

#Forecasting
prd <- predict(var, n.ahead = 10, ci = 0.95, dumvar = NULL)
print(prd)
plot(prd, "single")

Is this method sound?

Answer 1

I think you got it pretty right, but when building a VAR model, I usually make sure I follow these steps:

1. Select the variables

This is the most important part of building your model. If you want to forecast the price of an asset, you need to include variables that are related with the mechanism of price formation. The best way to do this is through a theoretical model. Since you did not mention what is the asset and what are the other variables you included in your model I really cannot say much about this item, but you can find a summary of asset pricing models in here.

2. Check the data and make the proper adjustments

Once you select the variables, you can make some adjustments to the data that will improve the estimation and interpretation of the model. It is useful to use summary statistics and see a plot of the series to detect outliers, missing data and other strange behaviors. When working with price data, people usually take natural logs, which is a variance-stabilizing transformation and also has a good interpretation (price difference in logs become continuously compound returns). I'm not sure if you have taken logs before estimating the model, but it is a good idea to do so if you are working with asset prices.

3. Check if data contains non-stationary components

Now you can use unit root tests to check if your series are stationary. If you are only interested in forecasting, as noted by @JacobH, you can run VAR in levels even when your series are non-stationary, but then your standard errors cannot be trusted, meaning that you can't make inference about the value of the coefficients. You've tested stationary using the ADF test, which is very commonly used in these applications, but note that you should specify if you want to run the test with i) no constant and no trend; ii) a constant and no trend; and iii) a constant and a trend. Usually price series have stochastic trends, so a linear trend will not be accurate. In this case you may choose the specification ii. In your code you used the ndiffs function of the forecast package. I am not sure which of those three alternatives this function implements in order to calculate the number of differences (I couldn't find it in the documentation). To check your result you may want to use the ur.df function in the "urca" package:

adf <- ur.df(x[, "VAR1"], type = "drift", lags = 10, selectlags = "AIC")

Note that this command will run the ADF test with a constant and the lags selected by the AIC command, with maximum lag of 10. If you have problems interpreting the results just look at this question. If the series are I(1) just use the difference, which will be equal to the continuously compounded returns. If the test indicates that the series are I(2) and you are in doubt about that you can use other tests, e.g. Phillips-Perron test (PP.test function in R). If all tests confirm that your series are I(2) (remember to use the log of the series before running the tests) then take the second difference, but note that your interpretation of the results will change, since now you are working with the difference of the continuously compounded returns. Prices of assets are usually I(1) since they are close to a random walk, which is a white noise when applying the first difference.

4. Select the order of the model

This can be done with commonly used criteria such as Akaike, Schwarz (BIC) and Hannan-Quinn. You've done that with the VARselect function and that is right, but remember what is the criterion that you used to make your decision. Usually different criteria indicate different orders for the VAR.

5. Check if there are cointegrating relationships

If all your series are I(1) or I(2), before running a VAR model, it is usually a good idea to check if there is no cointegration relationships between the series, specially if you want to make impulse response analysis with the residuals. You can do that using the Johansenn test or the Engle-Granger (only for bivariate models). In R you can run the Johansen test with the ca.jo function of the "urca" package. Note that this test also has different specifications. For price series I usually use the following code (where p is the lag length of item 4, performed with the series in levels):

jo_eigen <- ca.jo(x, type = "eigen", ecdet = "const", K = p)
jo_trace <- ca.jo(x, type = "trace", ecdet = "const", K = p)

6. Estimate the model

If your series are not cointegrated, you can easily estimate the model with the VAR command, as done in your code. In case the series are cointegrated you need to consider the long run relationship by estimating a Vector Error Correction model with the following code (where k is the order of cointegration):

vecm <- cajorls(joeigen, r = k)

7. Run diagnostics tests

To test if your model is well specified you can run a test of serial correlation on the residuals. In your code you used a Portmanteau test with the serial.test function. I've never used this function but I think it is OK. There is also a multivariate version of the Ljung-Box test implemented in the package MTS which you can run with the function mq.

8. Make predictions

After you are sure your model is well specified you can use the predict function as you did in your code. You can even plot impulse response functions to check how the variables respond to a particular shock using the irf function.

9. Evaluate predictions

Once you made your predictions you must evaluate them and compare against other models. Some methods to evaluate accuracy of forecasts can be found here, but to do that it is crucial that you divide your series in a training and a test set, as explained in the link.

Answer 2

I thought I would add to Regis A Ely very nice answer. His answer is not wrong, but using a VAR to forecast is different than using a VAR to do other VAR type things (i.e. IRF, FEVD, Historical Decomp. etc...). Consequently, some of the steps outlined by Regis A Ely will negatively effect your forecast in some cases.

Disclaimer:

When I refer to non-stationary data, I mean that the series contains a stochastic trend. If the data has a time/seasonal trend it must be filter appropriately.

First

Generally speaking, in an unrestricted VAR there is no need to worry about a spurious relationship. A spurious regression occurs when you regress a non-stationary series (Y) on another non-stationary series (X) and both series are not cointegrated. However, if you regress Y on X as well as lags of Y then the regression will not be spurious as the inclusion of the lag Y insures that the errors will be stationary. Said another way, lags of Y pick up the variation which was previous wrongly assigned to X. Since an unrestricted VAR is essentially a system of ARDL regressions where each equation contains the same number of lags and regressors, it should be clear that spurious regression are therefore not likely to be a problem. Said another way if your data is all I(1), regardless of whether of not it is co-integrated, you can run a VAR. VECM are only necessary when you want to both model and identify the short and long run/co-integration relationship between variables. The question now is, should you run the VAR in levels or in first differences.

Second

When forecasting, it is not necessary to first difference I(1) data. You can if you like, thought a surprisingly amount of practitioner don't. Remember when we have a non-stationary series, we can still obtain a consistent estimator. For a regression with a single lag of the dependent variable this is intuitive. If a series is following a random walk (i.e. non-stationary) we know the best estimate of where it will be next period is exactly were it was last period (i.e. beta is 1). The standard errors of estimates derived from models with non-stationary data, however, is different because strictly speaking the variance of the estimate approaches infinity as T approaches infinity. This, however, is not a problem for forecasting. Forecasting is essentially a conditional expectation and therefore only relies on the parameters estimates of your model and not standard errors. Further, prediction intervals of your forecast will either be obtained directly from your errors, by bootstrapping errors, or if you have a lot of data via empirical prediction intervals (my favorite!), all three of these approaches are unaffected by non-stationary data because again your errors will be stationary as per our spurious regression discussion above.

Why do I care?

The ADF test has low power, especially when the series is close to being unit root, but is not. Said another the ADF test will tend to mistakenly assert that a series is non-stationary when it in fact is not.

Assume that your ADF test wrongly assures that the series is non-stationary. If you make all the necessary transformation and estimate a VECM, your forecast is going to be wrong, because your model is wrong. This is why people forecast in levels.

What about Granger Causality???

You can even test GC with a VAR in levels when data is I(1). I know that sounds crazy. We know that inference is usually not possible with non-stationary data. It is however possible to test joint hypotheses, e.g. GC. This is shown in Toda and Yamamoto (1995) which draws on Sims, Stock and Watson (1990). For an application see http://davegiles.blogspot.com/2011/04/testing-for-granger-causality.html.

Last thing

If however, you want to use your VAR for things other than forecasting, be careful. A VAR in levels with non-stationary and co-integrated series can yield some weird results. For example, strictly speaking, the Moving Average representation of the VAR does not exist as the parameter matrix will not be invertible. Despite this fact IRF can still be obtained. Inference is also not feasible (thought joint hypotheses can be tested as discussed above).

Also be worry of small samples. Everything I've discussed works well in large sample, but things can get wacky in small samples. This is especially true for GC with I(1) data.