Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I am trying to find the relationship between S&P and six macroeconomic variables using vector error correction model (VECM).
What are the criteria for assessing the overall performance of the model?
I got low $R^2$ (see the output below), but this is OK for I(1) time series, I guess.
If you got low $R^2$ when the response variable is the first difference of an integrated variable (which I understand is the case), then it is as expected; if you had got that when the response variable were the level (rather than the first difference) of an integrated variable, that would have been less satisfactory.
As for "overall performance", I would do some residual diagnostics to make sure there are no pronounced ugly features such as large spikes/outliers, clear patterns across time, etc.
If you care about forecasting performance, try doing fair out-of-sample forecast evaluation. For example, use rolling windows to determine the model order and estimate the model in-sample and measure forecasting performance out of sample across the rolling windows. You may use root mean squared error or another measure you find relevant for forecast accuracy. You could also compare the performance of your model to that of some simpler benchmark model using the Diebold-Mariano test.
estat ic. $\endgroup$ – Dimitriy V. Masterov Mar 23 '16 at 21:15