Show Regression with Arima Errors Equivalent Form of Differenced Variables

How can you show that the regression $y_{t}=\beta_0 + \beta_1x_{t}+\eta_t$ where $\eta_t$ is arima(1,1,1) is equivalent to $y'_t = \beta_1x'_{t}+\eta'_t$ where $\eta'_t=\phi_1\eta'_{t-1}+e_t$ and $'$ denotes a first difference?

Also, is it possible to construct a model where everything is differenced except $\eta_t$, as in $y'_t = \beta_1x'_{t}+\eta_t$ where $\eta_t=\phi_1\eta_{t-1}+e_t$ where $'$ denotes a first difference or does the error term always have to be differenced if y or x is?

The model for $y_{t-1}$ is written as:

$$y_{t-1} = \beta_0 + \beta_1 x_{t-1} + \eta_{t-1} \,.$$

Thus, subtracting $y_t$ and $y_{t-1}$ gives:

\begin{eqnarray*} y_t - y_{t-1} \equiv y_t' &=& \beta_0 + \beta_1 x_t + \eta_t - (\beta_0 + \beta_1 x_{t-1} + \eta_{t-1}) \\ y_t' &=& \beta_1 x_t' + \eta_t' \,. \end{eqnarray*}

The error term $\eta_t$ follows an ARMA(1,1,1), therefore by definition we have:

$$\eta_t' = \phi \eta_{t-1}' + e_t + \theta e_{t-1} \,,$$

where $e_t$, $t=1,2,...,n$, are independent and identically distributed with mean $0$ and variance $\sigma^2$ (white noise). I think you either missed the term $\theta e_{t-1}$ or you meant that $\eta_t$ follows an ARIMA(1,1,0).

Your second question could be related to a particular case of an autoregressive distributed lag model (ADL). An ADL model of orders (1,1) can be defined as follows:

$$y_t = \beta_0 + \beta_1 y_{t-1} + \gamma_0 x_t + \gamma_1 x_{t-1} + u_t \,,$$

where the error term $u_t$ can follow a stationary ARMA process.

Different restrictions on the parameters, which could be tested by means of an F-test, lead to models with different interpretations (e.g. partial adjustment in the dependent variable to changes in the explanatory variables or adaptive expectations). For details, you may see Davidson and MacKinnon Section 13.4 or Econometric Methods with Applications in Business and Economics Section 7.5. (In a quick search, I did not find much information about this on wikipedia.)

If we set the restrictions $\beta_1 = 1$ and $\gamma_1 = -\gamma_0$, then we have a model in line with the model that you mention:

$$y_t' = \beta_0 + \gamma_0 x_t' + u_t \,.$$

In practice, if there is evidence of autocorrelation in the residuals of a fitted distributed lag model, then higher order lags of the dependent variable ($y_{t-2}, y_{t-3},...$) are usually added to the right hand side of the model so that there is no serial correlation in the error term. That is, $u_t$ is usually considered not autocorrelated.

The last reference cited above discusses the effect and consequences of different structures in the error term. For example, they warn that in the presence of a moving average in the error term the ordinary least squares estimator is not consistent (a maximum likelihood estimator should be used in that case).

• Very nice! And yes I meant to say arima (1,1,0). Your expanded answer was a nice bonus! – B_Miner Jul 10 '14 at 23:59