Due to lack of examples, I am having a hard time understanding the difference between a best predictor and a best linear predictor.

I think that $E[Y|X]$ is the best predictor for Y in terms of X, and if it happens to be linear, then it would also be the best linear predictor, (but please correct me if I'm wrong.)

So consider $Y=sin(X)+\epsilon$, where $X \sim U(0,2\pi)$ and $\epsilon \sim N(0, \sigma ^2)$. Then, I think that the best predictor would be $E[Y|X]=E[sin(X)+\epsilon|X]=E[sin(X)|X]+E[\epsilon|X]=E[sin(X)]=0$. (However I am not sure on the last step, I used $E[sin(X)]=\int_0^{2\pi} \frac{sin(x)}{2\pi}$.)

Since $E[Y|X]$ is not linear, then we would have to find the best linear predictor. I think to find the best linear predictor we would use $Y=E[Y]+\frac{ Cov(X,Y)}{Var(X)}(X-E[X])$. But I wanted to make sure before I started solving for the parameters. I got that $E[X]=\pi$, $E[Y]=0$, but I cannot find Cov(X,Y). If $E[Y|X]$ truly does equal 0 then I think Cov(X,Y) will also equal 0.

So I get that both the best predictor and linear predictor equal 0, and I don't think that is correct.

It is difficult for me to learn without being given examples so I am sorry in advanced for my many mistakes, thank you for your help.

  • $\begingroup$ Welcome to CV! Since you are new here, you may want to take a tour, which has information for new users. Is your question from a textbook? If yes, please add [self-study] tag and read its wiki. $\endgroup$ – Márcio Augusto Diniz Oct 24 '17 at 3:22
  • $\begingroup$ It's not, but thank you for your welcome! $\endgroup$ – Silvia Rossi Oct 24 '17 at 3:25
  • $\begingroup$ If you had $Y_i = \beta_0 + \beta_1 \sin(x_i) + \varepsilon_i$ where $\varepsilon_i$ was i.i.d Laplace$(0,\tau^2)$, say, or $t_3$, or uniform, the best linear estimator of $\underline\beta$ would be outperformed by some nonlinear estimator (different ones each time). $\endgroup$ – Glen_b Oct 24 '17 at 4:40

The best predictor is $$ \begin{eqnarray} E(Y|X) &=& E(sin(X) + \epsilon|X) \\ &=& E(sin(X)|X) + E(\epsilon|X) \\ &=& sin(X) + 0 \\ &=& sin(X) \end{eqnarray} $$ Notice that $sin(X)$ is a constant because $X$ is given in $E(sin(X)|X)$, i.e., $E(sin(X)) \neq E(sin(X)|X)$

Your rationale in the second part is correct. Only the calculation of $Cov(X, Y)$ should be reviewed: $$ \begin{eqnarray} Cov(X, Y) &=& Cov(X, sin(X) + \epsilon) \\ &=& Cov(X, sin(X)) + Cov(X, \epsilon) \\ &=& Cov(X, sin(X)) \\ &=& E([X - E(X)][sin(X) - E(sin(X))]) \\ &=& E([X - \pi][sin(X) - 0]) \\ &=& E([X - \pi]sin(X)) \\ &=& \int_0^{2\pi} [x - \pi]sin(X)\frac{1}{2\pi}dx \\ &=& \int_0^{2\pi} x sin(x)\frac{1}{2\pi}dx - \int_0^\frac{1}{2}sin(x)\frac{1}{2\pi}dx \\ &=& \int_0^{2\pi} x sin(x)\frac{1}{2\pi}dx \\ &=& \frac{1}{2\pi}\left(- x cos(x)|_0^{2\pi} + \int_0^{2\pi}cos(x)dx\right) \\ &=& \frac{1}{2\pi}\left(-2\pi + cos(x)|_0^{2\pi}\right) \\ &=& \frac{1}{2\pi}\left(-2\pi + 0\right) \\ &=& -1 \end{eqnarray} $$

Then, your best linear predictor is \begin{eqnarray} g(Y|X) &=& E(Y) + \frac{Cov(X, Y)}{Var(X)}(X - E(X)) \\ &=& 0 - \frac{3}{\pi^2}(X - \pi) \\ &=& -\frac{3}{\pi^2}(X - \pi) \end{eqnarray}

  • $\begingroup$ Thank you very much for your answer! Just to clarify, $E[Y|X]=0$ and not $sin(X)$? Thus the best predictor would be $E[Y|X]=0$ and the best linear predictor would be $-\frac{3}{\pi^2}(x-\pi)$? (I got that $var(x)=\frac{\pi^2}{3}$) Oh and I think $E[Y]=\epsilon$. $\endgroup$ – Silvia Rossi Oct 24 '17 at 4:15
  • $\begingroup$ I forgot the main part of your question. Let me edit it. $\endgroup$ – Márcio Augusto Diniz Oct 24 '17 at 4:25
  • $\begingroup$ $E(Y) = E(sin(X) + \epsilon) = E(sin(X)) + E(\epsilon) = 0 + 0 = 0$. Notice that $\epsilon$ is a random variable not given in $E(Y)$, therefore $E(\epsilon)$ has to be calculated using the fact that $\epsilon \sim N(0, \sigma^2)$. $\endgroup$ – Márcio Augusto Diniz Oct 24 '17 at 4:37
  • $\begingroup$ Oh right! Thank you, I forgot it wasn't simply a constant. Thank you so much for your help and time $\endgroup$ – Silvia Rossi Oct 24 '17 at 4:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.