# Prediction error for ARMA process

Let $$X(t)= \phi X_{t-12}+Z_t+\theta Z_{t-1}$$ where $$Z_t\sim WN(0,1)$$. I need to find prediction error for projecting $$X_t$$ onto $$H_{t-3}(X)$$ (Hilbert space).

So, I know that $$X_t \perp P_{H_{t-3}}X_t$$ what gives me:

$$\|X_t\|^2-\|P_{H_{t-3}}X_t\|^2=\langle \phi X_{t-12}+Z_t+\theta Z_{t-1}, \phi X_{t-12}+Z_t+\theta Z_{t-1}\rangle-\|P_{H_{t-3}}X_t\|^2=\sigma^2+\theta^2\sigma^2-\|P_{H_{t-3}}X_t\|^2$$

but what to do with $$\|P_{H_{t-3}}X_t\|^2$$?

Remark: Mind, that we can solve this only is $$(X_t)$$ is casual. A sufficient condition for that is $$|\varphi|<1$$. Using the representation with a lag operator $$B$$ $$$$(1-\varphi B^{12})X_t = (1 + \theta B)Z_t.$$$$ We can evaluate the casual form of $$(X_t)$$, which is $$$$X_t = (1+\theta B)\sum_{i = 0}^{\infty}\varphi^iX_{t-i}.$$$$ The sum does not converge when $$|\varphi| \geq 1$$.

Remark: You said, you knew that $$X_t\;\bot\;P_{H_{t-3}}X_t$$, which is not true in general. The truth is $$X_t - P_{H_{t-3}}X_t\;\bot\;P_{H_{t-3}}X_t$$ and it gives as: $$$$\text{Err}(P_{H_{t-3}}X_t) = ||X_t - P_{H_{t-3}}X_t||^2 = ||X_t||^2 - ||P_{H_{t-3}}X_t||^2.$$$$ The Pythagorean theorem "applies" here with $$a = X_t - P_{H_{t-3}}X_t$$, $$b = P_{H_{t-3}}X_t$$ and $$c = X_t$$, in its form $$a^2 = c^2 - b^2$$.

As we have casuality the problem goes much more easily - have a look at $$H_{t-3}$$ set. It contains only those elements that can be expressed as linear combinations of $$X_{t-3}, X_{t-4} \ldots X_{t-12} \ldots$$. Because of causality $$Z_{t}, Z_{t-1}, Z_{t-2}$$ are perpendicular to $$H_{t-3}$$, so the forecast is just $$$$P_{H_{t-3}}X_t = \varphi X_{t-12}.$$$$ You can imagine $$X_{t-12}$$ as only one element that the projection operator $$P_{H_{t-3}}$$ do not bring to zero (as it does with perpendicular elements). It works as in usual $$\mathbb{R}^3$$ space with perpendicular vectors to a projective space.

Another usefull fact is that the covariance function of $$(X_t)$$ is: $$$$\gamma_X(0) = \frac{\sigma^2(1 + \theta^2)}{1 - \varphi^2}.$$$$

Also we have $$\mathbb{E}X_t = m \in \mathbb{R}$$ and: $$$$\begin{split} ||X_t||^2 & = \langle X_t - m + m,\, X_t - m + m \rangle \\ & = \langle X_t - m,\,X_t - m \rangle + 2\langle X_t - m, \,m \rangle + \langle m,\,m\rangle \\ & = \gamma_X(0) + 2\mathbb{E}[(X_t - m)m] + m^2 \\ & = \gamma_X(0) + 2m\mathbb{E}X_t - 2m^2 + m^2 \\ & = \gamma_X(0) + m^2. \end{split}$$$$ But we can assume that $$m = 0$$ when we are considering projections. Simply center all variables in $$(X_t)$$ sequence, make a projection and shift them back by $$m$$, leaving an error untouched.

At this point, our problem is very simple: $$$$\begin{split} \text{Err}(P_{H_{t-3}}X_t) & = ||X_t||^2 - ||P_{H_{t-3}}X_t||^2 = \gamma_X(0) - ||\varphi X_{t-12}||^2 = \gamma_X(0) - |\varphi|^2 \gamma_X(0) \\ & = (1 - \varphi^2) \gamma_X(0) = (1 - \varphi^2) \frac{\sigma^2(1 + \theta^2)}{1 - \varphi^2} = \sigma^2(1 + \theta^2). \end{split}$$$$

• that is perfect, I really appreciate all the effort put into that solution. Thank you! Apr 11, 2021 at 14:54