Recently I have been reading through the Lecture-One course material that can be found at this link.

Anyways, in section-2 the author shows the following step in his derivation of an optimal guess by minimizing the function MSE(a):

enter image description here

where Y is a random variable and a is the prediction we are to make.

My question is: What is the properties/steps the author takes from making this jump. Specially, how is this step true?

The rest of the derivation is straightforward but I am finding this step hard to see and through all the literature I am parsing through, I am not finding anything that exactly works.

Thanks, MS

  • $\begingroup$ It's explained in equation (19) and the following paragraph. $\endgroup$ – Mark L. Stone May 15 '17 at 19:41
  • $\begingroup$ This is the Pythagorean Theorem. It becomes perfectly obvious when you consider the random variable $X=Y-a$, for then it merely asserts $$E(X^2)=E(X)^2+\operatorname{Var}(X).$$Subtract $E(X)^2$ from both sides to recognize the very definition of variance. $\endgroup$ – whuber May 15 '17 at 20:05

$$ E[(Y-a)^2] = E[(Y-E(Y) +E(Y)-a)^2] \\ =E[(Y-E(Y))^2] +2E[(Y-E(Y))(E(Y)-a)] + [E(Y)-a]^2\\ = Var(Y) + 2[E(Y)-E(Y)][E(Y)-a] + [E(Y)-a]^2\\ = Var[Y-a] + (E[Y-a])^2$$

  • $\begingroup$ You could have gone directly from the first expression to the last, because this simply amounts to the definition of variance. $\endgroup$ – whuber May 15 '17 at 20:50
  • $\begingroup$ For the illustration purpose, I think maybe it is better to write clearer. $\endgroup$ – user158565 May 16 '17 at 2:22
  • $\begingroup$ Understood and appreciated. I was implicitly suggesting that introducing superfluous algebraic steps can obfuscate the basic simplicity of a result, rather than clarify it. $\endgroup$ – whuber May 16 '17 at 13:03
  • $\begingroup$ The key step is the middle term $E(Y)-E(Y) = 0$. Generally, people do not recognize it. $\endgroup$ – user158565 May 16 '17 at 16:49
  • $\begingroup$ That is a little bit like saying that to go from the front of your house to the back, you should first travel out of the city, over the river, and then return by another route; and that the key step is crossing the river. I agree that may be correct: but highlighting the key step on an unnecessary detour is hardly a reason for taking that detour in the first place! $\endgroup$ – whuber May 16 '17 at 17:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.