I am building a VAR model to forecast the price of an asset and would like to know whether my method is statistically sound, whether the tests I have included are relevant and if more are needed to ensure a reliable forecast based on my input variables.

Below is my current process to check for Granger causality and forecast the selected VAR model.

require("forecast")
require("vars")

#Read Data
da=read.table("VARdata.txt", header=T)
dac <- c(2,3) # Select variables
x=da[,dac]

plot.ts(x)
summary(x)

#Run Augmented Dickey-Fuller tests to determine stationarity and differences to achieve stationarity.
ndiffs(x[, "VAR1"], alpha = 0.05, test = c("adf"))
ndiffs(x[, "VAR2"], alpha = 0.05, test = c("adf"))

#Difference to achieve stationarity
d.x1 = diff(x[, "VAR1"], differences = 2)
d.x2 = diff(x[, "VAR2"], differences = 2)

dx = cbind(d.x1, d.x2)
plot.ts(dx)

#Lag optimisation
VARselect(dx, lag.max = 10, type = "both")

#Vector autoregression with lags set according to results of lag optimisation. 
var = VAR(dx, p=2)

#Test for serial autocorrelation using the Portmanteau test
#Rerun var model with other suggested lags if H0 can be rejected at 0.05
serial.test(var, lags.pt = 10, type = "PT.asymptotic")

#ARCH test (Autoregressive conditional heteroscedasdicity)
arch.test(var, lags.multi = 10)

summary(var)

#Granger Causality test
#Does x1 granger cause x2?
grangertest(d.x2 ~ d.x1, order = 2)

#Does x2 granger cause x1?
grangertest(d.x1 ~ d.x2, order = 2)

#Forecasting
prd <- predict(var, n.ahead = 10, ci = 0.95, dumvar = NULL)
print(prd)
plot(prd, "single")

Is this method sound?

  • 1
    Are you using second differences? This is a little unusual and could make the model more sensitive than it needs to be. Also, may you expect cointegration in your system? And what about any deterministic time trends and/or seasonalities, have you checked for those? – Richard Hardy Jan 21 '16 at 19:55
  • @Richard, the differences to achieve stationarity are as far as I understand determined by the adf test, and would be adjusted according to its suggestion. Should the adf test determine it's stationary (return 0 I would not difference the variable). I have not thought of cointegration and seasonality, but was under the impression that the above method would take care of any trend in the variables. – youjustreadthis Jan 21 '16 at 20:04
  • 2
    ADF test is but a test, it comes with its limitations. Plotting the raw data, the first differences and eventually the second differences may be more informative than just running the test. Also, the ADF test has different specifications: (1) no constant, no trend; (2) constant, no trend; (3) constant and trend; lag order selection for the test also may be nontrivial. Hence, do not blindly rely on the results. From the subject-matter perspective, asset prices are usually integrated of order one, I(1). I(2) would be difficult to justify... – Richard Hardy Jan 21 '16 at 20:16
  • @youjustreadthis I've included an answer below. I strongly recommend you consider some of its implications – Jacob H Feb 14 '16 at 3:29
up vote 19 down vote accepted

I think you got it pretty right, but when building a VAR model, I usually make sure I follow these steps:

1. Select the variables

This is the most important part of building your model. If you want to forecast the price of an asset, you need to include variables that are related with the mechanism of price formation. The best way to do this is through a theoretical model. Since you did not mention what is the asset and what are the other variables you included in your model I really cannot say much about this item, but you can find a summary of asset pricing models in here.

2. Check the data and make the proper adjustments

Once you select the variables, you can make some adjustments to the data that will improve the estimation and interpretation of the model. It is useful to use summary statistics and see a plot of the series to detect outliers, missing data and other strange behaviors. When working with price data, people usually take natural logs, which is a variance-stabilizing transformation and also has a good interpretation (price difference in logs become continuously compound returns). I'm not sure if you have taken logs before estimating the model, but it is a good idea to do so if you are working with asset prices.

3. Check if data contains non-stationary components

Now you can use unit root tests to check if your series are stationary. If you are only interested in forecasting, as noted by @JacobH, you can run VAR in levels even when your series are non-stationary, but then your standard errors cannot be trusted, meaning that you can't make inference about the value of the coefficients. You've tested stationary using the ADF test, which is very commonly used in these applications, but note that you should specify if you want to run the test with i) no constant and no trend; ii) a constant and no trend; and iii) a constant and a trend. Usually price series have stochastic trends, so a linear trend will not be accurate. In this case you may choose the specification ii. In your code you used the ndiffs function of the forecast package. I am not sure which of those three alternatives this function implements in order to calculate the number of differences (I couldn't find it in the documentation). To check your result you may want to use the ur.df function in the "urca" package:

adf <- ur.df(x[, "VAR1"], type = "drift", lags = 10, selectlags = "AIC")

Note that this command will run the ADF test with a constant and the lags selected by the AIC command, with maximum lag of 10. If you have problems interpreting the results just look at this question. If the series are I(1) just use the difference, which will be equal to the continuously compounded returns. If the test indicates that the series are I(2) and you are in doubt about that you can use other tests, e.g. Phillips-Perron test (PP.test function in R). If all tests confirm that your series are I(2) (remember to use the log of the series before running the tests) then take the second difference, but note that your interpretation of the results will change, since now you are working with the difference of the continuously compounded returns. Prices of assets are usually I(1) since they are close to a random walk, which is a white noise when applying the first difference.

4. Select the order of the model

This can be done with commonly used criteria such as Akaike, Schwarz (BIC) and Hannan-Quinn. You've done that with the VARselect function and that is right, but remember what is the criterion that you used to make your decision. Usually different criteria indicate different orders for the VAR.

5. Check if there are cointegrating relationships

If all your series are I(1) or I(2), before running a VAR model, it is usually a good idea to check if there is no cointegration relationships between the series, specially if you want to make impulse response analysis with the residuals. You can do that using the Johansenn test or the Engle-Granger (only for bivariate models). In R you can run the Johansen test with the ca.jo function of the "urca" package. Note that this test also has different specifications. For price series I usually use the following code (where p is the lag length of item 4, performed with the series in levels):

jo_eigen <- ca.jo(x, type = "eigen", ecdet = "const", K = p)
jo_trace <- ca.jo(x, type = "trace", ecdet = "const", K = p)

6. Estimate the model

If your series are not cointegrated, you can easily estimate the model with the VAR command, as done in your code. In case the series are cointegrated you need to consider the long run relationship by estimating a Vector Error Correction model with the following code (where k is the order of cointegration):

vecm <- cajorls(joeigen, r = k)

7. Run diagnostics tests

To test if your model is well specified you can run a test of serial correlation on the residuals. In your code you used a Portmanteau test with the serial.test function. I've never used this function but I think it is OK. There is also a multivariate version of the Ljung-Box test implemented in the package MTS which you can run with the function mq.

8. Make predictions

After you are sure your model is well specified you can use the predict function as you did in your code. You can even plot impulse response functions to check how the variables respond to a particular shock using the irf function.

9. Evaluate predictions

Once you made your predictions you must evaluate them and compare against other models. Some methods to evaluate accuracy of forecasts can be found here, but to do that it is crucial that you divide your series in a training and a test set, as explained in the link.

  • Thank you very much for this detailed answer! With regards to the Johansen test for cointegration, does the implementation change when more than 2 variables are included? Believe I read that mulitcointegration carries pitfalls of its own. Sorry if this is better suited for a question of its own. – youjustreadthis Jan 25 '16 at 11:54
  • 1
    No, you can do it with the same code as above, but you may find more than one cointegrating vector in this case. The only limitation of this kind is with the Engle-Granger test, which is suitable only for bivariate series, but usually better in this case. – Regis A. Ely Jan 25 '16 at 12:06
  • 1
    This link may help to run and interpret johansenn tests. – Regis A. Ely Jan 25 '16 at 12:15
  • Alright, thanks! – youjustreadthis Jan 25 '16 at 12:18
  • Great job! I have edited some formatting and spelling though. Note that it is nice to have code pieces (even as small as function names) in backticks `, e.g. predict. Larger pieces of code can be formatted as code by selecting the text and clicking on the "quotes" button at the top of the editor window. – Richard Hardy Feb 1 '16 at 19:39

I thought I would add to Regis A Ely very nice answer. His answer is not wrong, but using a VAR to forecast is different than using a VAR to do other VAR type things (i.e. IRF, FEVD, Historical Decomp. etc...). Consequently, some of the steps outlined by Regis A Ely will negatively effect your forecast in some cases.

Disclaimer:

When I refer to non-stationary data, I mean that the series contains a stochastic trend. If the data has a time/seasonal trend it must be filter appropriately.

First

Generally speaking, in an unrestricted VAR there is no need to worry about a spurious relationship. A spurious regression occurs when you regress a non-stationary series (Y) on another non-stationary series (X) and both series are not cointegrated. However, if you regress Y on X as well as lags of Y then the regression will not be spurious as the inclusion of the lag Y insures that the errors will be stationary. Said another way, lags of Y pick up the variation which was previous wrongly assigned to X. Since an unrestricted VAR is essentially a system of seemingly unrelated regressions where each equation contains the same number of lags and regressors, it should be clear that spurious regression are therefore not likely to be a problem. Said another way if your data is all I(1), regardless of whether of not it is co-integrated, you can run a VAR. VECM are only necessary when you want to both model and identify the short and long run/co-integration relationship between variables. The question now is, should you run the VAR in levels or in first differences.

Second

When forecasting, it is not necessary to first difference I(1) data. You can if you like, thought a surprisingly amount of practitioner don't. Remember when we have a non-stationary series, we can still obtain a consistent estimator. For a regression with a single lag of the dependent variable this is intuitive. If a series is following a random walk (i.e. non-stationary) we know the best estimate of where it will be next period is exactly were it was last period (i.e. beta is 1). The standard errors of estimates derived from models with non-stationary data, however, is different because strictly speaking the variance of the estimate approaches infinity as T approaches infinity. This, however, is not a problem for forecasting. Forecasting is essentially a conditional expectation and therefore only relies on the parameters estimates of your model and not standard errors. Further, prediction intervals of your forecast will either be obtained directly from your errors, by bootstrapping errors, or if you have a lot of data via empirical prediction intervals (my favorite!), all three of these approaches are unaffected by non-stationary data because again your errors will be stationary as per our spurious regression discussion above.

Why do I care?

The ADF test has low power, especially when the series is close to being unit root, but is not. Said another the ADF test will tend to mistakenly assert that a series is non-stationary when it in fact is not.

Assume that your ADF test wrongly assures that the series is non-stationary. If you make all the necessary transformation and estimate a VECM, your forecast is going to be wrong, because your model is wrong. This is why people forecast in levels.

What about Granger Causality???

You can even test GC with a VAR in levels when data is I(1). I know that sounds crazy. We know that inference is usually not possible with non-stationary data. It is however possible to test joint hypotheses, e.g. GC. This is shown in Toda and Yamamoto (1995) which draws on Sims, Stock and Watson (1990). For an application see http://davegiles.blogspot.com/2011/04/testing-for-granger-causality.html.

Last thing

If however, you want to use your VAR for things other than forecasting, be careful. A VAR in levels with non-stationary and co-integrated series can yield some weird results. For example, strictly speaking, the Moving Average representation of the VAR does not exist as the parameter matrix will not be invertible. Despite this fact IRF can still be obtained. Inference is also not feasible (thought joint hypotheses can be tested as discussed above).

Also be worry of small samples. Everything I've discussed works well in large sample, but things can get wacky in small samples. This is especially true for GC with I(1) data.

  • 1
    Regarding First, could you back up your claim by a reference? I am not convinced by your argumentation. If $y_t$ and $x_t$ are independent random walks, a model $y_t=\beta_0+\beta_1 y_{t-1}+\dotsc+\beta_p y_{t-p}+\gamma x_t$ does not make sense as the left hand side diverges from the righ hand side. The question is, how bad is that for forecasting? If asymptotically $\hat\gamma^{OLS}$ gets close to zero (does it? where is the proof?), the problem gradually vanishes. But how large a sample is needed for that? Until a proof is given, I would continue avoiding spurious relationships. – Richard Hardy Feb 14 '16 at 11:51
  • 1
    Regarding Why do I care?, if the process has a root that is very close to a unit root, it behaves very much the same way as a unit-root process. When forecasting, there is thus little difference between assuming that the shocks are permanent and maintaining that they vanish extremely slowly. Unless you are forecasting very far into the future, the result will be virtually the same. That is why I am not too worried about the unit root test having low power for local alternatives. – Richard Hardy Feb 14 '16 at 11:57
  • 1
    Another small note regarding First: when talking about ADF test in Why do I care?, you say "your forecast is going to be wrong, because your model is wrong". Well, this applies to First as well, doesn't it? Forecasting using a model in which the left hand side diverges from the right hand side is indeed characterized by the above quote. – Richard Hardy Feb 14 '16 at 12:04
  • 1
    @Richardhardy For a proof of my first assertion see Chapter 18 in Hamilton 1994. In particular, section 18.2, Cures for Spurious Regression. It is worth noting that the OLS estimators is also efficient as they converge at a rate the sqrt of T. – Jacob H Feb 15 '16 at 4:24
  • 1
    @RichardHardy your second comment is a fair point. Thought my retort is that why spend time pre-testing and then filtering data only to potentially estimate the wrong model. For forecasting, the levels specification will generally be correct – Jacob H Feb 15 '16 at 5:02

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.