1
$\begingroup$

First, it's known that MSE can be decomposed into variance and bias^2 of the estimator:

enter image description here

But on many sources a 3rd term called "irreducible error" should also be a part of the decomposition, which is supposed to describe the noise in the population model/ true relationship.

For example:

$Err(x) = \left(E[\hat{f}(x)]-f(x)\right)^2 + E\left[\left(\hat{f}(x)-E[\hat{f}(x)]\right)^2\right] +\sigma_e^2$

However, the first proof says that the MSE is described completely by the variance and bias. How do I make sense of this seeming contradiction?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

The first case describes, say, the MSE of the OLS estimator $\hat\beta$ relative to the true value $\beta$ while the second describes the MSE of a prediction $\hat y_i=x_i\hat\beta$ made using the OLSE relative to the true $y_i$.

Improve this answer
$\endgroup$
4
  • $\begingroup$ I was also able to find a source that says: $MSE[ˆ g(t)] = [E(ˆ g(t)−g(t))]^2 + V ar(ˆ g(t))$. Derived from $MSE[ˆ g(t)] = E[(ˆ g(t)−g(t))^2]$. Nowhere is it stated that the noise is 0. Is this a mistake? $\endgroup$
    – Tony
    Commented Dec 9, 2016 at 13:02
  • $\begingroup$ That is tough to say without knowing what $g$ precisely is. Can you provide a reference? $\endgroup$
    – Christoph Hanck
    Commented Dec 9, 2016 at 13:03
  • $\begingroup$ unfortunately it's from a private course materials. But there really isn't anything more to it. g(t) is the fit and ^g(t) is its estimate. $\endgroup$
    – Tony
    Commented Dec 9, 2016 at 13:08
  • 1
    $\begingroup$ Maybe it was tacitly ignored as we cannot do anything about the irreducible part anyhow. $\endgroup$
    – Christoph Hanck
    Commented Dec 9, 2016 at 13:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.