Consider $\{X_t\}$ as a general time series including random variables $X_t$. Assume that we have observed $X$'s until time t. The goal is to come up with a function of the observed $X$'s to predict a future X, let's say unobserved $X_{t+h}$. Condsider $h(\vec{X}) = h((..., X_{t-1}, X_t))$ as our proposed function. We define mean prediction error as:

$$E\{(X_{t+h} - h(\vec{x}))^2 | \vec{x}\}$$

Goal: I want to show that among all possible functions h, the function $E(X_{t+h}|\vec{x})$ has the minimum mean prediction error.

Thank you for your help

  • $\begingroup$ What does "$c$" stand for in the definition of $h$? What is $Y$ and what connection, if any, does it have with $X_{t+h}$? In your question, what do you consider to be the arguments to the "function"--$Y$, $\vec{x}$, both? $\endgroup$
    – whuber
    Mar 18, 2013 at 21:30
  • 1
    $\begingroup$ Lemme do some edits...It now should make more sense $\endgroup$
    – Sam
    Mar 18, 2013 at 21:33

1 Answer 1


You can adapt the following argument to get a full answer to your question. For random variables $X$ and $Y$, and a measurable function $f$, we have $$ \newcommand{\E}{\mathbb{E}} \E[(Y-f(X))^2] = \E[(Y-\E[Y\mid X]+\E[Y\mid X]-f(X))^2] $$ $$ = \E[(Y-\E[Y\mid X])^2] - 2\,\E[(Y-\E[Y\mid X])(\E[Y\mid X]-f(X))] + \E[(\E[Y\mid X]-f(X))^2] \, . $$ Consider the middle term of the last expression. By definition, $\E[Y\mid X]-f(X)$ is a function (call it $g$) of $X$. Hence, using the properties of the conditional expectation, we have $$ \E[(Y-\E[Y\mid X])(\E[Y\mid X]-f(X))] = \E[(Y-\E[Y\mid X])g(X)] $$ $$ = \E[g(X)Y] - \E[g(X)\E[Y\mid X]] $$ $$ = \E[g(X)Y] - \E[\E[g(X)Y\mid X]] $$ $$ = \E[g(X)Y] - \E[g(X)Y] = 0 \, . $$ Therefore, $$ \E[(Y-f(X))^2] = \E[(Y-\E[Y\mid X])^2] + \E[(\E[Y\mid X]-f(X))^2] \, , $$ implying that $$ \E[(Y-f(X))^2] \geq \E[(Y-\E[Y\mid X])^2] \, , $$ with equality if we choose $f(X)=\E[Y\mid X]$ a.s. $\newcommand{\cF}{\mathscr{F}}$

Your proof follows mutatis mutandis from this one. Define $\cF_t=\sigma(X_{t-k}:k\geq 0)$. Let $Z$ be $\cF_t$-measurable. Use what we just did to prove that $$ \E[(X_{t+k}-Z)^2\mid\cF_t] \geq \E[(X_{t+k}-\E[X_{t+k}\mid\cF_t])^2\mid\cF_t] $$ holds a.s. The result follows.

  • 1
    $\begingroup$ Do you know of good textbooks or other references where I can learn "properties of the conditional expectation"? I feel like all of my books either (1) only cover the basics, or (2) are extremely advanced and abstract (measure theory). I would like to learn more about topics like: conditional distributions, conditional expectations, expectations that contain matrices & vectors, multivariate normal distributions, copulas, etc. $\endgroup$
    – Joe
    Oct 11, 2020 at 16:57
  • 1
    $\begingroup$ Hi Joe. Try Grimmett and Stirzaker and do a lot of problems. amazon.com/Probability-Random-Processes-Geoffrey-Grimmett/dp/… $\endgroup$
    – Zen
    Oct 11, 2020 at 22:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.