2
$\begingroup$

I need to use exogenous variables for my time series forecasting.

And I found that I can include my exogenous variables into my ARIMA model using exogenous option.

I want to know how this option works.

Statsmodels docoument(https://www.statsmodels.org/stable/generated/statsmodels.tsa.arima_model.ARIMA.html) says

" If exogenous variables are given, then the model that is fit is

$ϕ(L)(y_t−X_tβ)=θ(L)ϵ_t$

where ϕ and θ are polynomials in the lag operator, L. This is the regression model with ARMA errors, or ARMAX model. "

I cannot understand what this formula means.

I googled it and found what lag operator is.

But I still don't understand what the above formula implies.

I'm a undergraduate and took regression course.

Could you explain above formula in terms of regression?

$\endgroup$
5
  • $\begingroup$ Hi: can you get a hold of andew harvey's "econometric analysis of time series". if so, he gives the econometric perspective on arima models with exogenous variables which I think is quite enlightening. note that it's not an easy book but worth the effort. $\endgroup$
    – mlofton
    Commented Nov 14, 2019 at 12:59
  • $\begingroup$ Note that the statsmodel documentation calls this both "regression model with ARMA errors" and "ARMAX model". These are different models, and as described here this is a regression with ARMA errors, as explained below in the answer by @CloseToC. The line about ARMAX is an error in the statsmodel documentation. $\endgroup$
    – Chris Haug
    Commented Nov 14, 2019 at 15:26
  • $\begingroup$ $$\phi(L) y_t = \phi(L) X_t \beta + \theta(L) \epsilon_t$$ I thought ARMAX refers to any ARMA model with exogenous variables, e.g.en.wikipedia.org/wiki/… $\endgroup$
    – Josef
    Commented Nov 14, 2019 at 19:59
  • $\begingroup$ That's also what I thought. Also: Hyndman says (robjhyndman.com/hyndsight/arimax) there is little between the two variants of ARMA + X in terms of forecasting, but regression with ARMA error as implemented in statsmodels has a much better interpretation $\endgroup$
    – CloseToC
    Commented Nov 15, 2019 at 9:52
  • $\begingroup$ @CloseToC That's correct. The fact that the interpretation (and parameters) is different is precisely why calling it "an ARMAX model" is an error in the documentation. I mentioned it because if the OP decided that they wanted to get more information about "what the formula means" and looked up "ARMAX model", they would get a completely different and incorrect answer. $\endgroup$
    – Chris Haug
    Commented Nov 15, 2019 at 15:42

1 Answer 1

3
$\begingroup$

The ARMA(p,q) model has $p$ lags of the dependent variable and an error term that is a moving average of $q$ lags. In standard regression notation the model is:

$$y_t = \phi_1 y_{t-1} + ... + \phi_p y_{t-p} + \epsilon_t - \theta_1 \epsilon_{t-1} -...-\theta_q \epsilon_{t-q}$$

With the backshift/lag operator $L$ notation, you can write eg $y_{t-2}$ as as $L(L(y_t)) = L^2 y_{t} $and so you can write the model as $$(1 - \phi_1 L - .... - \phi_p L^p) y_t = (1 - \theta_1 L - ... - \theta_q L^q)\epsilon_t$$. You can make this more compact by replacing the polynomials of the lag operator: $$\phi(L) y_t = \theta(L) \epsilon_t$$

So what is $$\phi(L) (y_t - X_t \beta) = \theta(L) \epsilon_t$$?

It's $$y_t - X_t \beta - \phi_1 (y_{t-1} - X_{t-1}\beta) - ... - \phi_p (y_{t-p} - X_{t-p}\beta) = \epsilon_t - \theta_1 \epsilon_{t-1} -...-\theta_q \epsilon_{t-q}$$ or $$y_t = X_t \beta + \phi_1 (y_{t-1} - X_{t-1}\beta) + ... +\phi_p (y_{t-p} - X_{t-p}\beta) + \epsilon_t - \theta_1 \epsilon_{t-1} -...-\theta_q \epsilon_{t-q}$$

Finally, if you define the regression residual as $u_t = y_t - X_t \beta$ you can write this as in an easy to interpret way as:

$$y_t = X_t \beta + \phi_1 u_t + ... +\phi_p u_{t-p} + \epsilon_t - \theta_1 \epsilon_{t-1} -...-\theta_q \epsilon_{t-q} = X_t \beta + n_t$$

And that's simply regression with an ARMA error $n_t$ as the statsmodel documentation says.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.