# What's the model representation for the first difference of a local level model?

This is my first exercise for space state models and I've a few questions I'd need to resolve before I actually start doing the exercise. Unfortunately, I'm self teaching (I have no professor to ask) and I'm afraid there's no solution companion for Durbin and Koopman (2012)!

Exercise 2.13.1 from Time Series Analysis by State Space Methods Second Edition

Consider the local level model (2.3).

(a) Give the model representation for $$x_t = y_t - y_{t-1}$$, for $$t = 2, ..., n$$.

(b) Show that the model for $$x_t$$ in (a) can have the same statistical properties as the model given by $$x_t = \epsilon_t + \theta \epsilon_{t-1}$$ where $$\epsilon \sim N(0, \sigma_{\epsilon}^2)$$ are independent disturbances with variance $$\sigma_{\epsilon}^2 > 0$$ and for some value $$\theta$$.

(c) For what value of $$\theta$$, in terms of $$\sigma_{\epsilon}^2$$ and $$\sigma_{\eta}^2$$, are the model representations for $$x_t$$ in (a) and (b) equivalent? Comment.

For the record, the local level model (2.3) is given by:

$$y_t = \alpha_t + \epsilon_t \quad\quad \epsilon_t \sim N(0, \sigma_{\epsilon}^2)$$

$$\alpha_{t+1} = \alpha_t + \eta_t \quad\quad \eta_t \sim N(0, \sigma_{\eta}^2)$$

First of all, the model proposed in (a) looks like noise (which makes perfect sense since it's the first difference of a random walk). Is the following representation correct?

$$x_t = y_t - y_{t-1} = \alpha_t + \epsilon_t - \alpha_{t-1} - \epsilon_{t-1}$$ $$x_t = \alpha_{t-1} + \eta_{t-1} + \epsilon_t - \alpha_{t-1} - \epsilon_{t-1}$$ $$x_t = \eta_{t-1} + \epsilon_{t} - \epsilon_{t-1}$$

This makes me doubt. First, state disturbance $$\eta_{t-1}$$ is now part of the observation equation. Second, what does the state equation mean now that the observation equation doesn't relate to the unobserved states $$\alpha_t$$? Third, and somehow related, what's the mean unobserved state now that $$\alpha_t$$ isn't anymore on the formula? Zero?

Additionally, I wonder how to show that models have the same statistical properties. What do you have to prove to say they're the same? Same expected value and variance of observation $$x_t$$, unobserved state $$\alpha_t$$, prediction error $$v_t = x_t - a_t$$, filtered unobserved state, updated unobserved state, etc.? Since all random variables are Normal, I guess showing the first two moments match is enough, but a) what distribution (marginal, conditionals, conditionals on what?) of b) what variables (observed, hidden state, prediction error, etc.) should be equal?

Any comment is much appreciated!

Update

This is where I got after the hints provided by @Glen_b and @javlacalle.

(a)

$$x_t = \eta_{t-1} + \epsilon_t - \epsilon_{t-1}$$

(b)

Respect to model $$x_t$$ given in (a)

$$E[x_t | x_{t-1}] = 0$$ $$\gamma(0) = Var(x_t | x_{t-1}) = \sigma_{\eta}^2 + 2\sigma_{\epsilon}^2$$ $$\gamma(1) = Cov(x_t, x_{t-1}) = -\sigma_{\epsilon}^2$$ $$\gamma(2) = Cov(x_t, x_{t-2}) = 0$$ $$\rho(1) = \frac{-\sigma_{\epsilon}^2}{\sigma_{\eta}^2 + 2\sigma_{\epsilon}^2}$$ $$\rho(2) = 0$$

Respect to model $$x_t$$ proposed in (b), which I renamed to $$z_t$$ to avoid confusion

$$E[z_t | z_{t-1}] = 0$$ $$\gamma(0) = Var(z_t | z_{t-1}) = \sigma_{\epsilon}^2 (1 + \theta^2)$$ $$\gamma(1) = Cov(z_t, z_{t-1}) = \theta \sigma_{\epsilon}^2$$ $$\gamma(2) = Cov(z_t, z_{t-2}) = 0$$ $$\rho(1) = \frac{\theta}{1 + \theta^2}$$ $$\rho(2) = 0$$

(c)

$$E[x_t | x_{t-1}] = E[z_t | z_{t-1}] = 0 \quad \qquad (c.1)$$

$$\gamma_{x_t}(0) = \gamma_{z_t}(0) \leftrightarrow \sigma_{\eta}^2 + 2\sigma_{\epsilon}^2 = \sigma_{\epsilon}^2 (1 + \theta^2) \quad \quad (c.2)$$

$$\gamma_{x_t}(1) = \gamma_{z_t}(1) \leftrightarrow -\sigma_{\epsilon}^2 = \theta \sigma_{\epsilon}^2 \rightarrow \theta = -1 \quad \quad (c.3)$$

$$\gamma_{x_t}(2) = \gamma_{z_t}(2) = 0 \quad \quad (c.4)$$

$$\rho_{x_t}(1) = \rho_{z_t}(1) \leftrightarrow \frac{-\sigma_{\epsilon}^2}{\sigma_{\eta}^2 + 2\sigma_{\epsilon}^2} = \frac{\theta}{1 + \theta^2} \quad \quad (c.5)$$

$$\rho_{x_t}(2) = \rho_{z_t}(2) = 0 \quad \quad (c.6)$$

Equations c.1, c.4 and c.6 imply no restrictions for $$\theta$$, but equations c.2, c.3 and c.5 are clearly not consistent.

• @Glen_b Thank you for pointing out the omission, I edited the question. – mugen Jan 21 '15 at 15:09
• Your equation (c.2), (c.3) and (c.5) seem to be consistent, but implying that $\sigma^2_\eta=0$ and $\theta=-1$. But this, in terms of $q$ as in the answer by @javlacalle, means $q=0$ and not $q>0$... – hejseb Jan 22 '15 at 9:58

You have arrived to the stationary form of the local level model:

$$\Delta y_t \equiv x_t = \underbrace{\Delta \alpha_t}_{\eta_{t-1}} + \Delta \epsilon_t \,,$$

where $\Delta$ is the difference operator such that $\Delta y_t = y_t - y_{t-1}$.

Now, I think it is easier to first check the statistical properties (mean, covariances, autocorrelations) of this stationary form.

For example, the mean of this process is given by:

$$\hbox{E}[x_t] = \hbox{E}[\eta_{t-1}] + \hbox{E}[\epsilon_t] - \hbox{E}[\epsilon_{t-1}] = 0 + 0 - 0 = 0 \,.$$

You can do the same to obtain the covariances of order $k$, $\gamma(k)$:

\begin{eqnarray} \begin{array}{ll} \gamma(0) &=& E\left[(\eta_{t-1} + \epsilon_t - \epsilon_{t-1})^2\right] = \dots \\ \gamma(1) &=& E\left[(\eta_{t-1} + \epsilon_t - \epsilon_{t-1})(\eta_{t-2} + \epsilon_{t-1} - \epsilon_{t-2})\right] &=& \dots \\ \gamma(2) &=& \cdots \\ \gamma(>2) &=& \cdots \end{array} \end{eqnarray}

You just need to take the expectation of the cross-products of all terms bearing in mind that $\eta_t$ and $\epsilon_t$ are independently distributed, they are independent of each other and the variance of each one are respectively $\sigma^2_\eta$ and $\sigma^2_\epsilon$.

Then, it will be straightforward to get the expression of the autocorrelations of order $k>0$, $\rho(k) = \frac{\gamma(k)}{\gamma(0)}$. This will have a form that is characteristic of a moving-average of order 1, MA(1) (the autocorrelations are zero for $k>1$) and, hence, $x_t$ can be represented as a MA(1) process and $y_t$ as an ARIMA(0,1,1) process.

In order to find out the relationship between the parameters of the local level model and the MA coefficient, you can equate the expression of the first order autocorrelation obtained before with the expression of the first order autocorrelation of a MA(1). Following the same strategy as above, you can find that $\rho(1)$ for a MA(1) with coefficient $\theta$ is given by $\rho(1) = \theta/(1 + \theta^2)$. The expression that you get by doing this will also reveal that the local level model is a restricted ARIMA(0,1,1) model where the MA coefficient $\theta$ can take only negative values.

Edit

Equation (c.5) is okay. You can get the relationship between the parameters of the local level model and the MA coefficient solving the equation (c.5) for $\theta$. You can rewrite it as a quadratic equation to be solved for $\theta$. One of the solutions can be discarded because it implies a non-invertible MA, $|\theta|>1$.

When solving this equation, it will be helpful to define $q=\sigma^2_\eta/\sigma^2_\epsilon$. Also, check that $\frac{\sqrt{\sigma^4_\eta + 4\sigma^2_\eta\sigma^2_\epsilon}}{2\sigma^2_\epsilon} = \frac{\sqrt{q^2 + 4q}}{2}$. This way you will get a more neat expression. Then, given that $0 < q < \infty$, you can check that the range of possible values for $\theta$ are zero or negative values.

• Thanks for your help, I could get a better idea of what I should been doing after reading your hints a couple times. I edited the question with more details, and now I only need to compute the restrictions for $\theta$, which are inconsistent so far. – mugen Jan 22 '15 at 2:42
• Thanks for your new comments, they look clear so far. I'll try to finish it tonight, hope this time I can finally tackle the exercise! – mugen Jan 22 '15 at 14:39

Some explicit guidance and hints:

In (b) you would either need to go on and show the properties of the series $x_t$ (what's its ACF, say? What are the properties you need?) or to explicitly rewrite it in the form of an MA (which is easier, I think - you might just recast it as a transform to an MA in $\zeta$ say, $x_t=\theta(B)\,\zeta_t$, where $\zeta_t=...$).
• First of all, thanks for your time. 1) Since my state equation is null, should I remove too the state noise $\eta_{t-1}$? This noise comes from the state equation of the original model, and I don't see how I'd remove it. 2) Model (a) is MA(1) if I could remove state disturbance from the original model $\eta_{t-1}$ and I realise the model given in (b) is a plain MA(1), does this mean all I'd need is to cast model (a) as a MA(1) process and show they share the same variance $\sigma_{\epsilon}^2? – mugen Jan 21 '15 at 15:16 • No,$\eta$is still in the process, but you moved it to the observation equation. You can't eliminate it from both, and you don't need to. – Glen_b Jan 21 '15 at 15:30 • ok, thank you for your input. I'll try to give it a try again tonight, when I get home.$\eta_{t-1}$is the state disturbance for the original process$y_t$and ended naturally in the observation equation for the new process$x_t$, I'll try to work this out from this observation equation$x_t = \eta_{t-1} + \epsilon_t - \epsilon_{t-1}$, having a null state equation$\alpha_t = 0$(without noise) for the process. – mugen Jan 21 '15 at 15:37 • Following your advice, I left the state noise from the original process$y_t$in the observation equation from the new process$x_t\$. Also, I computed expectation, autocovariance and autocorrelation for both processes to show they're statistically the same. Finally, I tried to find the conditions for equivalence, but constraints seem to be inconsistent. Looks like I have to improve something in the last part finish it. – mugen Jan 22 '15 at 2:46