I am using linear regression.

$Y_i = \beta_0 + \beta_1 x_i + \varepsilon_i$

$\varepsilon_i \overset{iid}{\sim} Normal(0, \sigma^2)$

At $X = x^\ast$, let's define the mean response as

$\mu^\ast = \beta_0 + \beta_1 x^\ast$.

The estimator for the mean response is

$\hat{\mu^\ast} = \hat{\beta_0} + \hat{\beta_1} x^\ast$.

Since $E(\hat{\mu^\ast}) = E(\hat{\beta_0} + \hat{\beta_1} x^\ast) = \beta_0 + \beta_1 x^\ast = \mu^\ast$, $\hat{\mu^\ast}$ is unbiased.

Now, let's define $Y^\ast$ as an individual response at $x^\ast$. This individual response deviates from the mean response by $\varepsilon^\ast$. Thus,

$Y^\ast = y^\ast + \varepsilon^\ast$.

Now, I think that the estimator is this:

$\hat{Y^\ast} = \hat{y^\ast} + \hat{\varepsilon^\ast}$.

Is this estimator unbiased? I'm not sure. If we use the MSE as $\hat{\varepsilon^\ast}$, then

$E(\hat{Y^\ast}) = E(\hat{\beta_0} + \hat{\beta_1} x^\ast + \hat{\varepsilon^\ast}) = \beta_0 + \beta_1 x^\ast = \mu^\ast$,

which is CLEARLY not $Y^\ast$.

Thus, is this estimator truly biased, or did I make a mistake with my math?


1 Answer 1


What does $ \hat{\varepsilon^\ast}$ mean?
Since $ \varepsilon^\ast$ has mean $0$, doesn't it make sense that the best prediction is $0$?

  • 1
    $\begingroup$ As I wrote in my question, I'm using mean squared error for $\hat{\varepsilon^\ast}$. $\endgroup$ Feb 7, 2021 at 2:13
  • $\begingroup$ In the last calculation you use the fact that its expected value is 0. But, I still don’t know what it is. Do you have a formula for it? Does it depend on any of the observed data or depend on $x*$? $\endgroup$
    – John L
    Feb 7, 2021 at 17:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.