Suppose we have the simple set up

$Y = \alpha X + \epsilon$

Where $X$ and $\epsilon$ are both mean zero, variance one and independent. $\alpha$ is a constant.

Trivially, $E[Y|X] = \alpha X$.

By simple linear regression and Gauss-Markov theorem I think that

$E[X|Y] = \frac{Cov(X,Y)}{Var(Y)} Y = \frac{\alpha}{1+\alpha^2} Y$

But how can I constructively derive this result? In particular in such a way that the same logic follows for vector $X$ and $\alpha$ (i.e. multiple regression)

  • 2
    $\begingroup$ Well, the result holds when $X$ and $Y$ are jointly Gaussian, but in general,, it is not true that $E[X\mid Y]$, the minimum mean-square error estimator of $X$ given $Y$ is the same as the (constrained) linear minimum mean-square error estimator $\frac{\operatorname{cov}(X,Y)}{\operatorname{var}(Y)}Y$. So you cannot get a "constructive" derivation of the result you seek unless you specify in more details what your assumptions are. $\endgroup$ – Dilip Sarwate Feb 13 '16 at 2:09
  • 1
    $\begingroup$ @DilipSarwate aha, thanks, so that was my mistake, they have to be jointly Gaussian to make progress. That result is "merely" the best linear estimator. $\endgroup$ – Korone Feb 15 '16 at 8:21
  • 1
    $\begingroup$ See this answer of mine for a way to arrive at the result. $\endgroup$ – Dilip Sarwate Feb 15 '16 at 13:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.