0
$\begingroup$

Consider the mean squared error $$\text{MSE}(x_0) = E_{T}(f(x_0)-\hat{y}_0)^{2}$$ $$ = E_{T}(\hat{y}_0-E_{T}(\hat{y}_0))^{2}+(f(x_0)-E_{T}(\hat{y}_0)^{2})$$

Is the first term after the equal sign in the second line basically saying the following: Get an estimate and find its deviation from the average estimates over all training samples. The estimate $\hat{y}_0$ will depend on the training set which is why we take the expected value.

Also is $E_{T}(\hat{y}_0)$ constant?

$\endgroup$
1
  • 1
    $\begingroup$ You probably mean $MSE(y_0)$, not of $x_0$. Also, not sure what the expectation $E_T$ stands for -- what is the probability space over which you are integrating? Is your $x$ random or fixed? $\endgroup$
    – StasK
    Commented Jun 14, 2013 at 15:01

1 Answer 1

4
$\begingroup$

Your intuition seems right, but for clarity, I would rewrite your equations as follows,

\begin{align} \text{MSE}(\hat{y}) &= E\left[(y-\hat{y})^2\right] \\ &= \underbrace{E\left[(\hat{y}-E[\hat{y}])^2\right]}_{\text{Var}[\hat{y}]} + \underbrace{(y-E[\hat{y}])^2}_{\text{Bias}^2} \end{align}

Here $y$ is the actual value of whatever we are estimating, and $\hat{y}$ is its estimator (the value of $\hat{y}$ will depend on the data).

The first term is the variance of the estimator (which is more or less what you said). Also, $E[\hat{y}]$ is non-random (this may be what you meant by constant).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.