# Difference between best predictor and best linear predictor

Due to lack of examples, I am having a hard time understanding the difference between a best predictor and a best linear predictor.

I think that $E[Y|X]$ is the best predictor for Y in terms of X, and if it happens to be linear, then it would also be the best linear predictor, (but please correct me if I'm wrong.)

So consider $Y=sin(X)+\epsilon$, where $X \sim U(0,2\pi)$ and $\epsilon \sim N(0, \sigma ^2)$. Then, I think that the best predictor would be $E[Y|X]=E[sin(X)+\epsilon|X]=E[sin(X)|X]+E[\epsilon|X]=E[sin(X)]=0$. (However I am not sure on the last step, I used $E[sin(X)]=\int_0^{2\pi} \frac{sin(x)}{2\pi}$.)

Since $E[Y|X]$ is not linear, then we would have to find the best linear predictor. I think to find the best linear predictor we would use $Y=E[Y]+\frac{ Cov(X,Y)}{Var(X)}(X-E[X])$. But I wanted to make sure before I started solving for the parameters. I got that $E[X]=\pi$, $E[Y]=0$, but I cannot find Cov(X,Y). If $E[Y|X]$ truly does equal 0 then I think Cov(X,Y) will also equal 0.

So I get that both the best predictor and linear predictor equal 0, and I don't think that is correct.

It is difficult for me to learn without being given examples so I am sorry in advanced for my many mistakes, thank you for your help.

• Welcome to CV! Since you are new here, you may want to take a tour, which has information for new users. Is your question from a textbook? If yes, please add [self-study] tag and read its wiki. – Márcio Augusto Diniz Oct 24 '17 at 3:22
• It's not, but thank you for your welcome! – Silvia Rossi Oct 24 '17 at 3:25
• If you had $Y_i = \beta_0 + \beta_1 \sin(x_i) + \varepsilon_i$ where $\varepsilon_i$ was i.i.d Laplace$(0,\tau^2)$, say, or $t_3$, or uniform, the best linear estimator of $\underline\beta$ would be outperformed by some nonlinear estimator (different ones each time). – Glen_b Oct 24 '17 at 4:40

## 1 Answer

The best predictor is $$\begin{eqnarray} E(Y|X) &=& E(sin(X) + \epsilon|X) \\ &=& E(sin(X)|X) + E(\epsilon|X) \\ &=& sin(X) + 0 \\ &=& sin(X) \end{eqnarray}$$ Notice that $sin(X)$ is a constant because $X$ is given in $E(sin(X)|X)$, i.e., $E(sin(X)) \neq E(sin(X)|X)$

Your rationale in the second part is correct. Only the calculation of $Cov(X, Y)$ should be reviewed: $$\begin{eqnarray} Cov(X, Y) &=& Cov(X, sin(X) + \epsilon) \\ &=& Cov(X, sin(X)) + Cov(X, \epsilon) \\ &=& Cov(X, sin(X)) \\ &=& E([X - E(X)][sin(X) - E(sin(X))]) \\ &=& E([X - \pi][sin(X) - 0]) \\ &=& E([X - \pi]sin(X)) \\ &=& \int_0^{2\pi} [x - \pi]sin(X)\frac{1}{2\pi}dx \\ &=& \int_0^{2\pi} x sin(x)\frac{1}{2\pi}dx - \int_0^\frac{1}{2}sin(x)\frac{1}{2\pi}dx \\ &=& \int_0^{2\pi} x sin(x)\frac{1}{2\pi}dx \\ &=& \frac{1}{2\pi}\left(- x cos(x)|_0^{2\pi} + \int_0^{2\pi}cos(x)dx\right) \\ &=& \frac{1}{2\pi}\left(-2\pi + cos(x)|_0^{2\pi}\right) \\ &=& \frac{1}{2\pi}\left(-2\pi + 0\right) \\ &=& -1 \end{eqnarray}$$

Then, your best linear predictor is \begin{eqnarray} g(Y|X) &=& E(Y) + \frac{Cov(X, Y)}{Var(X)}(X - E(X)) \\ &=& 0 - \frac{3}{\pi^2}(X - \pi) \\ &=& -\frac{3}{\pi^2}(X - \pi) \end{eqnarray}

• Thank you very much for your answer! Just to clarify, $E[Y|X]=0$ and not $sin(X)$? Thus the best predictor would be $E[Y|X]=0$ and the best linear predictor would be $-\frac{3}{\pi^2}(x-\pi)$? (I got that $var(x)=\frac{\pi^2}{3}$) Oh and I think $E[Y]=\epsilon$. – Silvia Rossi Oct 24 '17 at 4:15
• I forgot the main part of your question. Let me edit it. – Márcio Augusto Diniz Oct 24 '17 at 4:25
• $E(Y) = E(sin(X) + \epsilon) = E(sin(X)) + E(\epsilon) = 0 + 0 = 0$. Notice that $\epsilon$ is a random variable not given in $E(Y)$, therefore $E(\epsilon)$ has to be calculated using the fact that $\epsilon \sim N(0, \sigma^2)$. – Márcio Augusto Diniz Oct 24 '17 at 4:37
• Oh right! Thank you, I forgot it wasn't simply a constant. Thank you so much for your help and time – Silvia Rossi Oct 24 '17 at 4:39