Conditional Expectation has the minimum mean prediction error

Question

Consider $\{X_t\}$ as a general time series including random variables $X_t$. Assume that we have observed $X$'s until time t. The goal is to come up with a function of the observed $X$'s to predict a future X, let's say unobserved $X_{t+h}$. Condsider $h(\vec{X}) = h((..., X_{t-1}, X_t))$ as our proposed function. We define mean prediction error as:

$$E\{(X_{t+h} - h(\vec{x}))^2 | \vec{x}\}$$

Goal: I want to show that among all possible functions h, the function $E(X_{t+h}|\vec{x})$ has the minimum mean prediction error.

Thank you for your help

Answer 1

You can adapt the following argument to get a full answer to your question. For random variables $X$ and $Y$, and a measurable function $f$, we have $$ \newcommand{\E}{\mathbb{E}} \E[(Y-f(X))^2] = \E[(Y-\E[Y\mid X]+\E[Y\mid X]-f(X))^2] $$ $$ = \E[(Y-\E[Y\mid X])^2] - 2\,\E[(Y-\E[Y\mid X])(\E[Y\mid X]-f(X))] + \E[(\E[Y\mid X]-f(X))^2] \, . $$ Consider the middle term of the last expression. By definition, $\E[Y\mid X]-f(X)$ is a function (call it $g$) of $X$. Hence, using the properties of the conditional expectation, we have $$ \E[(Y-\E[Y\mid X])(\E[Y\mid X]-f(X))] = \E[(Y-\E[Y\mid X])g(X)] $$ $$ = \E[g(X)Y] - \E[g(X)\E[Y\mid X]] $$ $$ = \E[g(X)Y] - \E[\E[g(X)Y\mid X]] $$ $$ = \E[g(X)Y] - \E[g(X)Y] = 0 \, . $$ Therefore, $$ \E[(Y-f(X))^2] = \E[(Y-\E[Y\mid X])^2] + \E[(\E[Y\mid X]-f(X))^2] \, , $$ implying that $$ \E[(Y-f(X))^2] \geq \E[(Y-\E[Y\mid X])^2] \, , $$ with equality if we choose $f(X)=\E[Y\mid X]$ a.s. $\newcommand{\cF}{\mathscr{F}}$

Your proof follows mutatis mutandis from this one. Define $\cF_t=\sigma(X_{t-k}:k\geq 0)$. Let $Z$ be $\cF_t$-measurable. Use what we just did to prove that $$ \E[(X_{t+k}-Z)^2\mid\cF_t] \geq \E[(X_{t+k}-\E[X_{t+k}\mid\cF_t])^2\mid\cF_t] $$ holds a.s. The result follows.