I cannot understand formula for exogenous options in statsmodels' ARIMA

Question

I need to use exogenous variables for my time series forecasting.

And I found that I can include my exogenous variables into my ARIMA model using exogenous option.

I want to know how this option works.

Statsmodels docoument(https://www.statsmodels.org/stable/generated/statsmodels.tsa.arima_model.ARIMA.html) says

" If exogenous variables are given, then the model that is fit is

$ϕ(L)(y_t−X_tβ)=θ(L)ϵ_t$

where ϕ and θ are polynomials in the lag operator, L. This is the regression model with ARMA errors, or ARMAX model. "

I cannot understand what this formula means.

I googled it and found what lag operator is.

But I still don't understand what the above formula implies.

I'm a undergraduate and took regression course.

Could you explain above formula in terms of regression?

Answer 1

The ARMA(p,q) model has $p$ lags of the dependent variable and an error term that is a moving average of $q$ lags. In standard regression notation the model is:

$$y_t = \phi_1 y_{t-1} + ... + \phi_p y_{t-p} + \epsilon_t - \theta_1 \epsilon_{t-1} -...-\theta_q \epsilon_{t-q}$$

With the backshift/lag operator $L$ notation, you can write eg $y_{t-2}$ as as $L(L(y_t)) = L^2 y_{t} $and so you can write the model as $$(1 - \phi_1 L - .... - \phi_p L^p) y_t = (1 - \theta_1 L - ... - \theta_q L^q)\epsilon_t$$. You can make this more compact by replacing the polynomials of the lag operator: $$\phi(L) y_t = \theta(L) \epsilon_t$$

So what is $$\phi(L) (y_t - X_t \beta) = \theta(L) \epsilon_t$$?

It's $$y_t - X_t \beta - \phi_1 (y_{t-1} - X_{t-1}\beta) - ... - \phi_p (y_{t-p} - X_{t-p}\beta) = \epsilon_t - \theta_1 \epsilon_{t-1} -...-\theta_q \epsilon_{t-q}$$ or $$y_t = X_t \beta + \phi_1 (y_{t-1} - X_{t-1}\beta) + ... +\phi_p (y_{t-p} - X_{t-p}\beta) + \epsilon_t - \theta_1 \epsilon_{t-1} -...-\theta_q \epsilon_{t-q}$$

Finally, if you define the regression residual as $u_t = y_t - X_t \beta$ you can write this as in an easy to interpret way as:

$$y_t = X_t \beta + \phi_1 u_t + ... +\phi_p u_{t-p} + \epsilon_t - \theta_1 \epsilon_{t-1} -...-\theta_q \epsilon_{t-q} = X_t \beta + n_t$$

And that's simply regression with an ARMA error $n_t$ as the statsmodel documentation says.