6
$\begingroup$

I know from standard theory that the bias-variance decomposition for Mean Squared Error is (for an estimator $\hat{\mu}$ of $\mu$):

$$ E\left[\left(\hat{\mu}-\mu\right)^2\right] = Var(\hat{\mu}) + \left(E\left[\mu\right]-\mu\right)^2 = Var(\hat{\mu}) +Bias(\hat{\mu}, \mu)^2 $$

However, if today we are talking about vectors, where $\boldsymbol{\hat{\mu}}$ is an estimator of $\boldsymbol{\mu}$, both of which are $n\times 1$ vectors, I was wondering if there is a corresponding nice decomposition as in the scalar case above for:

$$ E\left[||\boldsymbol{\hat{\mu}}-\boldsymbol{\mu}||^2\right] $$ ?

Thanks!

$\endgroup$

1 Answer 1

7
$\begingroup$

Simply note that

$$|| \widehat{\mu} - \mu ||^2 = \sum\limits_{i = 1}^{n} (\widehat{\mu}_{i} - \mu_{i})^2$$

Then, the answer is given by the decomposition you gave earlier:

$$ \mathbb{E}[(\widehat{\mu}_{i} - \mu_{i})^2] = Var[\widehat{\mu}_{i}] + [Bias(\widehat{\mu}_{i}, \mu_{i} )]^2 $$

Summing all up, we get

$$ \mathbb{E}[||\widehat{\mu} - \mu||^2] = \sum\limits_{i = 1}^{n} Var[\widehat{\mu}_{i}] + [Bias(\widehat{\mu}_{i}, \mu_{i} )]^2 $$

Another issue, totally different, is the covariance matrix $\mathbb{E}[(\widehat{\mu} - \mu)(\widehat{\mu} - \mu)^t]$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.