5
$\begingroup$

I have a time series with a level shift. Thus, when treating it with an ARIMA model, I use arima(1,0,0)+xreg. The xreg is a dummy variable for the level shift. And then I use linear regression: $$ y(t)=β_0 + β_1y(t-1) + β_3{\rm levelshift} + μ(t), $$ where ${\rm levelshift}$ is the same as xreg in ARIMA.

I think these two models are exactly the same, however, eviews gives the different results even if I use least square methods for both estimations.

Could anyone explain that? Is the different results due to xreg or y(t-1)?

$\endgroup$
5
$\begingroup$

The formulation of an ARIMA model with exogenous regressors is not generally the same as a linear regression model with lagged dependent variables. To my knowledge, the formulation in software packages for the ARIMA model with exogenous regressors is the following:

$$ \left[ y(t) - \beta_0 - \beta_3 \hbox{levelshift}(t) \right] = \beta_1 \left[y(t-1) - \beta_0 - \beta_3 \hbox{levelshift}(t-1)\right] + \mu(t) \,, $$

which, as you can see, differs from the regression equation that you give. This issue is sometimes a bit misleading. Below, I give some details based on a larger discussion that I give here.


Linear regression model with lagged dependent variables

$$y_t = \beta_0 + \beta_1 x_{1,t} + \cdots + \beta_k x_{k,t} + \phi_1 y_{t-1} + \cdots + \phi_p y_{t-p} + \epsilon_t \,, \quad \epsilon_t \sim NID(0, \sigma^2) \,.$$

  • The coefficient $\beta_1$ measures how the dependent variable $y_t$ changes when there is a unit change in $x_1$.
  • The role of the lagged dependent variables is usually to whiten the residuals, i.e. remove serial correlation in the disturbance term in order to gain efficiency in the Ordinary Least Squares estimates. This is for example used in the so-called augmented Dickey-Fuller regression.
  • $\beta_0$ is an intercept, the expected value of $y_t$ when $x_{1,t}$ is zero.

ARMA time series model with exogenous regressors

Although the above formulation could be understood as an AR(p) model with regressors, this model is actually specified a follows (and, to my knowledge, this is how it is implemented, this is how it is implemented in software packages):

$$(y_t - \beta_0 - \beta_1 x_{1,t}) = \sum_{i=1}^p \phi_i (y_{t-i} - \beta_0 - \beta_1 x_{1,t-i}) + \epsilon_t \,, \quad \epsilon_t \sim NID(0, \sigma^2) \,.$$

To save space, I have used only on regressor $x_1$. The MA term is not included, as the comparison with the linear regression model is not straightforward in that case.

  • The role of the lagged variables (and of moving average terms of a general ARMA model) is to capture the overall dynamics observed in the data, e.g. looking to the autocorrelation function.
  • In the absence of exogenous regressors, $\beta_0$ is not an intercept as in the regression model above, it is instead the mean of $y_t$. With explanatory variables, the mean of the series is not constant and changes with $x_t$.

Edit (In response to the comment by @habu.)

Both specifications are correct and can be estimated for example by maximum likelihood. Which one is more convenient depends on the context.

In a regression analysis, the interpretation of the coefficients related to the explanatory variables is more natural in terms of how the dependent variable changes with a unit increase in one of the regressors. In this context we don't care much about the value of the coefficients related to the lags of the dependent variable since they are included just to render the residuals uncorrelated.

In time series analysis, we are usually more interested in knowing the overall dynamics in the level of the series rather than how a regressor explains the dependent variable. Also, in this context, the coefficients related to the lags are interesting since they contain relevant information about the dynamics of the data (e.g. autocorrelations, periodicity of the most important cycles in the series).

Apart from these differences in the interpretation and purposes, both equations are valid and estimable.

$\endgroup$
  • $\begingroup$ (+1) for the insight on AR implementation. Is there, for the sake of argument, anything fundamentally incorrect (i.e. violating the theoretical principles of the method), should one choose to implement any of the AR family of models as an LR model with lagged variables? My reading of the equations above leads me to think that the key difference implementation-wise would be that, while a lagged-variable LR formulation could be OLS-estimated using linear optimization, the standard AR implementations would require non-linear optimization techniques. Am I missing something here? $\endgroup$ – habu May 5 '15 at 16:52
  • 1
    $\begingroup$ @habu Both specifications are correct and can be estimated for example by maximum likelihood. Which one is more convenient depends on the context. I have edited my answer with some comments about this. $\endgroup$ – javlacalle May 5 '15 at 17:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.