Alternative construction of ARMA(1,1) process

Question

My question is related to the exercise 2.9, p. 79 in Brockwell & Davis, An Introduction to Time Series Analysis and Forecasting, 2nd edition, New-York, Springer, 2002 (It is also related to exercise 3.5, same reference).

Let {$Y_t$} be a process defined by $$ Y_t = X_t + W_t,$$ where $\{W_t\}\sim \mbox{WN}(0, \sigma_w^2),$ and {$X_t$} is the following AR(1) process $$ X_t - \phi X_{t-1}= Z_t,\quad \{Z_t\}\sim \mbox{WN}(0, \sigma_z^2),$$ and $E(W_s Z_t)=0$ for all $s$ and $t$. The goal of this exercise is to show that $Y_t$ is in fact an ARMA(1,1) process. We define the process $\{U_t\}$ as $$U_t= Y_t - \phi Y_{t-1}$$ 1) We compute the autocovariance function of $U_t$ at lag $h$ and we get $$\gamma_U(h) = \left\{ \begin{array}{ll} \displaystyle \sigma^2_z + \sigma_w^2 (1+\phi^2) , & \text{ if } h=0, \\ \displaystyle -\phi\ \sigma^2_w ,& \text{ if } |h|=1, \\ \displaystyle 0, & \text{ if } |h|>1. \end{array} \right. $$ $\{U_t\}$ is 1-correlated and hence is a MA(1) process (by Proposition 2.1.1, B & D).

2) Thus, there exists a white noise sequence $\{\varepsilon_t\}$ with variance $\sigma_\varepsilon^2$ such that: $$Y_t - \phi Y_{t-1} = U_t = \varepsilon_t + \lambda \varepsilon_{t-1}. $$ Then we want to express the parameters characterizing the MA(1) process $\{U_t\}$, namely $\lambda$ and $\sigma_\varepsilon^2$, in terms of the parameters characterizing $\{Y_t\}$ and $\{X_t\}$, namely, $\phi$, $\sigma_w^2$ and $\sigma^2_z$.

By equalizing the autocovariance function of the two representations, we obtain the following system: $$ \left\{ \begin{array}{rcl} \displaystyle \sigma^2_\varepsilon (1+\lambda^2) &= & \sigma^2_z + \sigma_w^2 (1+\phi^2), \\ \displaystyle \lambda \sigma_\varepsilon^2 & = & -\phi\ \sigma^2_w. \\ \end{array} \right. $$ If $\phi = 0$, we get $\lambda = 0 $ and the process $\{Y_t\}$ is a white noise with variance $\sigma_\varepsilon^2 = \sigma_z^2 + \sigma_w^2$. We now assume that $\phi \neq 0$ and $\lambda \neq 0$. Dividing the two equations of the system, we get: $$ \frac{1+\lambda^2}{\lambda} = \frac{1}{-\phi} \frac{\sigma^2_z}{\sigma^2_w} -\frac{1+\phi^2}{\phi} \Leftrightarrow \frac{1+\lambda^2}{\lambda} = -\frac{k^2 + \phi^2 +1 }{\phi} . $$ where $k^2 = \frac{\sigma^2_z}{\sigma^2_w}$. We then get the following second order equation for $\lambda$: $$\phi \lambda^2 + (k^2 + \phi^2 +1)\lambda + \phi. $$ The latter equations admits two real (and positive) solutions, if I am not wrong.

Question: is there any issue with the non-identifiability of the MA(1) process defined by $ \varepsilon_t + \lambda \varepsilon_{t-1}$? In other words, is that correct that I have, for the same process $\{Y_t\}$, two solutions for representing it in this way?

Answer 1

Rather than working through the auto-covariance function, it is simpler to perform the analysis as an algebraic exercise working with the initial recursive equations for the two levels of the model. Taking time back by one unit in the upper process gives the equation:

$$Y_{t-1} = X_{t-1} + W_{t-1}.$$

Multiplying both sides by $\phi$ and substituting the recursions from the lower and upper processses then gives:

$$\begin{aligned} \phi Y_{t-1} &= \phi X_{t-1} + \phi W_{t-1} \\[6pt] &= X_{t} - Z_t + \phi W_{t-1} \\[6pt] &= Y_{t} - W_{t} - Z_t + \phi W_{t-1}. \\[6pt] \end{aligned}$$

Re-arranging this equation gives the recursive form:

$$Y_{t} = \phi Y_{t-1} + W_{t} + Z_{t} - \phi W_{t-1}.$$

Now, to put this in the desired form, let $\{ \varepsilon_t \} \sim \text{N}(0, \sigma_\varepsilon^2)$ denote a new noise process for the model and write the model in ARMA(1,1) form as:

$$Y_{t} = \phi Y_{t-1} + \varepsilon_t + \lambda \varepsilon_{t-1}.$$

Equating the auto-covariance functions for the two forms and combining the equations gives the resulting equation you derived in your question:

$$\frac{1+\lambda^2}{\lambda} = -\frac{\sigma_z^2 / \sigma_w^2 + \phi^2 + 1}{\phi},$$

which can be written as the quadratic form:

$$\phi \lambda^2 + \lambda (\sigma_z^2 / \sigma_w^2 + \phi^2 + 1) + \phi = 0.$$

This quadratic form has two solutions, but only one is inside the unit circle (giving an invertible MA part). The equation for this solution is:

$$\lambda = - \frac{(\sigma_z^2 / \sigma_w^2 + \phi^2 + 1) - \sqrt{(\sigma_z^2 / \sigma_w^2 + \phi^2 + 1)^2 - 4 \phi^2}}{2 \phi}.$$

Answer 2

Here is my answer, feel free to comment and or correct. By assuming the invertibility of the process, I can discard one of the two solutions of the equation in $\lambda$.

We remind the second order equation for $\lambda$: $$\phi \lambda^2 + (k^2 + \phi^2 +1)\lambda + \phi. $$ The discriminant of this equation is simply given by: $$\Delta = (k^2 + \phi^2 +1)^2 - 4\phi^2 = (k^2 + \phi^2 +1 - 2\phi) (k^2 + \phi^2 +1 + 2\phi)$$ $$ 1+k^2 - 4(1+k^2) = -3(1+k^2)$$ % $$(1 + k^2 - 2 \phi + \phi^2) (1 + k^2 + 2 \phi + \phi^2).$$ Both factors of the discriminant are strictly positive. Indeed, the polynomial $$ \phi^2 - 2\phi + k^2 +1 $$ does not admit any root and is hence strictly positive since its own discriminant is given by $$ 4 - 4(1+k^2) = -4k^2 <0.$$ Hence, $\lambda$ finally admits two solutions: \begin{align*} \lambda_1 & = - \frac{1 + k^2 + \phi^2 - \sqrt{(k^2 + \phi^2 +1)^2 - 4\phi^2}}{2 \phi}, \\ \lambda_2 & = - \frac{1 + k^2 + \phi^2 + \sqrt{(1 + k^2 + \phi^2)^2-4 \phi^2 }}{2 \phi}. \end{align*} Note that we have $$(1 + k^2 + \phi^2)^2 > (1 + k^2 + \phi^2)^2 - 4\phi^2 \Leftrightarrow 1 + k^2 + \phi^2 > \sqrt{(1 + k^2 + \phi^2)^2 - 4\phi^2}, $$ and thus both solutions $\lambda_1$ and $\lambda_2$ are of opposite sign of $\phi$ since the numerator of the expression of $\lambda_1$ and $\lambda_2$ is positive, which is compliant with the fact that the variance $\sigma_\varepsilon$, given by, $\sigma_\varepsilon^2 = \frac{\phi\ \sigma_w^2}{\lambda}$, must be positive. We know that the process will be invertible if and only if $|\lambda|< 1$. We will show that $|\lambda_1| \leq1$ and $|\lambda_2|\geq 1$.

Consider now a right-angled triangle with hypotenuse $1+k^2+ \phi^2$, and a cathetus of length $2|\phi|$. Then, by the Pythagorean theorem, the last cathetus has length $\sqrt{(1 + k^2 + \phi^2)^2 - 4\phi^2}$. By the triangle inequality, we have: \begin{eqnarray*} & & 2|\phi| + \sqrt{(1 + k^2 + \phi^2)^2 - 4\phi^2} \geq 1 + k^2 + \phi^2\\ & \Leftrightarrow & 1+ k^2 + \phi^2 - \sqrt{(1 + k^2 + \phi^2)^2 - 4\phi^2} \leq 2|\phi|\\ &\Leftrightarrow &\frac{1 + k^2 + \phi^2 - \sqrt{(1 + k^2 + \phi^2)^2 - 4\phi^2}}{2|\phi|}\leq 1\\ &\Leftrightarrow & |\lambda_1| = \frac{1 + k^2 + \phi^2 - \sqrt{(1 + k^2 + \phi^2)^2 - 4\phi^2}}{2|\phi|}\leq 1. \end{eqnarray*} On the other hand, note that $$ \sqrt{(1 + k^2 + \phi^2)^2 - 4\phi^2} \geq 0 \Rightarrow \sqrt{(1 + k^2 + \phi^2)^2 - 4\phi^2} \geq -[(1-|\phi|)^2 + k^2]. $$ Then, by some simple computations, we get: \begin{eqnarray*} & & \sqrt{(1 + k^2 + \phi^2)^2 - 4\phi^2} \geq -[(1-|\phi|)^2 + k^2]\\ & \Leftrightarrow & \sqrt{(1 + k^2 + \phi^2)^2 - 4\phi^2} \geq -[1-2|\phi| + \phi^2 + k^2]\\ & \Leftrightarrow & 1 + k^2 + \phi^2 + \sqrt{(1 + k^2 + \phi^2)^2 - 4\phi^2} \geq 2|\phi| \\ &\Leftrightarrow & |\lambda_2| = \frac{1 + k^2 + \phi^2 + \sqrt{(1 + k^2 + \phi^2)^2 - 4\phi^2}}{2|\phi|}\geq 1. \end{eqnarray*} Hence, $\lambda_2$ does not yield a invertible solution to the process defined above and we retain $\lambda_1 := \lambda$ as the unique invertible solution.