3
$\begingroup$

I am trying to estimate the effect of an external regressor $x_t$ on a time series $y_t$. My first attempt was using an ARIMAX(p,d,q) Model to estimate $\beta_x$ while controlling for the ARMA structure in the errors.

In a second attempt, I modeled $y_t$ using a linear OLS model which includes $y_{t-1}, y_{t-2}, ..., y_{t-p}$ as lagged predictors. The estimates I am getting for $\beta_x$ are quite different between the two approaches.

From a theoretical point, the OLS model with lagged predictors should produce similar results as an AR-model (neglecting optimizations issues). Hence the difference must lay in the MA-part of the ARIMA model (neglecting the differentiation for a moment).

So on a general level, under which conditions should I expect to obtain biased estimates of $x_t$ from a linear OLS model?

Best, W.

$\endgroup$
  • $\begingroup$ The difference and/or similarity between the models might be less obvious than you might think. Check out Rob J. Hyndman's blog post "The ARIMAX model muddle". $\endgroup$ – Richard Hardy Jun 20 '16 at 10:20
2
$\begingroup$

The difference and/or similarity between the models might be less obvious than you might think. Check out Rob J. Hyndman's blog post "The ARIMAX model muddle".

My first attempt was using an ARIMAX(p,d,q) Model to estimate $\beta_x$ while controlling for the ARMA structure in the errors.

Already here we have a slight confusion: ARIMAX is not the same as regression with ARMA errors. If you are using, for example, arima function in R, you are fitting the latter. The model then looks like

\begin{aligned} y_t &= \beta_0 + \beta_1 x_t + u_t \\ u_t &= \varphi_1 u_{t-1} + \dotsc + \varphi_p u_{t-p} + \varepsilon_t + \theta_1 \varepsilon_{t-1} + \dotsc + + \theta_q \varepsilon_{t-q} \end{aligned}

In a second attempt, I modeled $y_t$ using a linear OLS model which includes $y_{t−1}, y_{t−2},\dotsc,y_{t−p}$ as lagged predictors.

That should look something like below

$$ y_t = \beta_0 + \beta_1 x_t + \gamma_1 y_{t-1} + \dotsc + \gamma_p y_{t-p} + v_t. $$

Intuitively, if $x_t$ is correlated with a linear combination of the other regressors (lagged $y$s), inclusion of those regressors will change $\beta_1$, potentially a lot. Meanwhile, if $u_t$ is relatively well behaved, $\beta_1$ will be more or less what you would expect it to be in a simple regression $y_t=\beta_0+\beta_1 x_t+\epsilon_t$ with $\epsilon_t \sim i.i.d.(0,\sigma^2)$. This "intuition" is quite loose, though, and I don't know whether expecting $u_t$ to be well behaved is justified. For a reliable treatment, see the blog post cited above.

Also, your argumentation

From a theoretical point, the OLS model with lagged predictors should produce similar results as an AR-model (neglecting optimizations issues). Hence the difference must lay in the MA-part of the ARIMA model (neglecting the differentiation for a moment)

becomes tangential once we recognize how different the two models actually are.

$\endgroup$
  • $\begingroup$ Thanks for the clarification, the link is indeed helpful. $\endgroup$ – winwin Jul 5 '16 at 7:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.