Let $X(t)= \phi X_{t-12}+Z_t+\theta Z_{t-1}$ where $Z_t\sim WN(0,1)$. I need to find prediction error for projecting $X_t$ onto $H_{t-3}(X)$ (Hilbert space).

So, I know that $X_t \perp P_{H_{t-3}}X_t$ what gives me:

$\|X_t\|^2-\|P_{H_{t-3}}X_t\|^2=\langle \phi X_{t-12}+Z_t+\theta Z_{t-1}, \phi X_{t-12}+Z_t+\theta Z_{t-1}\rangle-\|P_{H_{t-3}}X_t\|^2=\sigma^2+\theta^2\sigma^2-\|P_{H_{t-3}}X_t\|^2$

but what to do with $\|P_{H_{t-3}}X_t\|^2$?


1 Answer 1


Remark: Mind, that we can solve this only is $(X_t)$ is casual. A sufficient condition for that is $|\varphi|<1$. Using the representation with a lag operator $B$ \begin{equation} (1-\varphi B^{12})X_t = (1 + \theta B)Z_t. \end{equation} We can evaluate the casual form of $(X_t)$, which is \begin{equation} X_t = (1+\theta B)\sum_{i = 0}^{\infty}\varphi^iX_{t-i}. \end{equation} The sum does not converge when $|\varphi| \geq 1$.

Remark: You said, you knew that $X_t\;\bot\;P_{H_{t-3}}X_t$, which is not true in general. The truth is $X_t - P_{H_{t-3}}X_t\;\bot\;P_{H_{t-3}}X_t$ and it gives as: \begin{equation} \text{Err}(P_{H_{t-3}}X_t) = ||X_t - P_{H_{t-3}}X_t||^2 = ||X_t||^2 - ||P_{H_{t-3}}X_t||^2. \end{equation} The Pythagorean theorem "applies" here with $a = X_t - P_{H_{t-3}}X_t$, $b = P_{H_{t-3}}X_t$ and $c = X_t$, in its form $a^2 = c^2 - b^2$.

As we have casuality the problem goes much more easily - have a look at $H_{t-3}$ set. It contains only those elements that can be expressed as linear combinations of $X_{t-3}, X_{t-4} \ldots X_{t-12} \ldots $. Because of causality $Z_{t}, Z_{t-1}, Z_{t-2}$ are perpendicular to $H_{t-3}$, so the forecast is just \begin{equation} P_{H_{t-3}}X_t = \varphi X_{t-12}. \end{equation} You can imagine $X_{t-12}$ as only one element that the projection operator $P_{H_{t-3}}$ do not bring to zero (as it does with perpendicular elements). It works as in usual $\mathbb{R}^3$ space with perpendicular vectors to a projective space.

Another usefull fact is that the covariance function of $(X_t)$ is: \begin{equation} \gamma_X(0) = \frac{\sigma^2(1 + \theta^2)}{1 - \varphi^2}. \end{equation}

Also we have $\mathbb{E}X_t = m \in \mathbb{R}$ and: \begin{equation} \begin{split} ||X_t||^2 & = \langle X_t - m + m,\, X_t - m + m \rangle \\ & = \langle X_t - m,\,X_t - m \rangle + 2\langle X_t - m, \,m \rangle + \langle m,\,m\rangle \\ & = \gamma_X(0) + 2\mathbb{E}[(X_t - m)m] + m^2 \\ & = \gamma_X(0) + 2m\mathbb{E}X_t - 2m^2 + m^2 \\ & = \gamma_X(0) + m^2. \end{split} \end{equation} But we can assume that $m = 0$ when we are considering projections. Simply center all variables in $(X_t)$ sequence, make a projection and shift them back by $m$, leaving an error untouched.

At this point, our problem is very simple: \begin{equation} \begin{split} \text{Err}(P_{H_{t-3}}X_t) & = ||X_t||^2 - ||P_{H_{t-3}}X_t||^2 = \gamma_X(0) - ||\varphi X_{t-12}||^2 = \gamma_X(0) - |\varphi|^2 \gamma_X(0) \\ & = (1 - \varphi^2) \gamma_X(0) = (1 - \varphi^2) \frac{\sigma^2(1 + \theta^2)}{1 - \varphi^2} = \sigma^2(1 + \theta^2). \end{split} \end{equation}

  • $\begingroup$ that is perfect, I really appreciate all the effort put into that solution. Thank you! $\endgroup$
    – thesecond
    Apr 11, 2021 at 14:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.