# Conditional Expectation has the minimum mean prediction error

Consider $\{X_t\}$ as a general time series including random variables $X_t$. Assume that we have observed $X$'s until time t. The goal is to come up with a function of the observed $X$'s to predict a future X, let's say unobserved $X_{t+h}$. Condsider $h(\vec{X}) = h((..., X_{t-1}, X_t))$ as our proposed function. We define mean prediction error as:

$$E\{(X_{t+h} - h(\vec{x}))^2 | \vec{x}\}$$

Goal: I want to show that among all possible functions h, the function $E(X_{t+h}|\vec{x})$ has the minimum mean prediction error.

• What does "$c$" stand for in the definition of $h$? What is $Y$ and what connection, if any, does it have with $X_{t+h}$? In your question, what do you consider to be the arguments to the "function"--$Y$, $\vec{x}$, both?
– whuber
Mar 18, 2013 at 21:30
• Lemme do some edits...It now should make more sense
– Sam
Mar 18, 2013 at 21:33

You can adapt the following argument to get a full answer to your question. For random variables $X$ and $Y$, and a measurable function $f$, we have $$\newcommand{\E}{\mathbb{E}} \E[(Y-f(X))^2] = \E[(Y-\E[Y\mid X]+\E[Y\mid X]-f(X))^2]$$ $$= \E[(Y-\E[Y\mid X])^2] - 2\,\E[(Y-\E[Y\mid X])(\E[Y\mid X]-f(X))] + \E[(\E[Y\mid X]-f(X))^2] \, .$$ Consider the middle term of the last expression. By definition, $\E[Y\mid X]-f(X)$ is a function (call it $g$) of $X$. Hence, using the properties of the conditional expectation, we have $$\E[(Y-\E[Y\mid X])(\E[Y\mid X]-f(X))] = \E[(Y-\E[Y\mid X])g(X)]$$ $$= \E[g(X)Y] - \E[g(X)\E[Y\mid X]]$$ $$= \E[g(X)Y] - \E[\E[g(X)Y\mid X]]$$ $$= \E[g(X)Y] - \E[g(X)Y] = 0 \, .$$ Therefore, $$\E[(Y-f(X))^2] = \E[(Y-\E[Y\mid X])^2] + \E[(\E[Y\mid X]-f(X))^2] \, ,$$ implying that $$\E[(Y-f(X))^2] \geq \E[(Y-\E[Y\mid X])^2] \, ,$$ with equality if we choose $f(X)=\E[Y\mid X]$ a.s. $\newcommand{\cF}{\mathscr{F}}$
Your proof follows mutatis mutandis from this one. Define $\cF_t=\sigma(X_{t-k}:k\geq 0)$. Let $Z$ be $\cF_t$-measurable. Use what we just did to prove that $$\E[(X_{t+k}-Z)^2\mid\cF_t] \geq \E[(X_{t+k}-\E[X_{t+k}\mid\cF_t])^2\mid\cF_t]$$ holds a.s. The result follows.