6
$\begingroup$

I know from standard theory that the bias-variance decomposition for Mean Squared Error is (for an estimator $\hat{\mu}$ of $\mu$):

$$ E\left[\left(\hat{\mu}-\mu\right)^2\right] = Var(\hat{\mu}) + \left(E\left[\mu\right]-\mu\right)^2 = Var(\hat{\mu}) +Bias(\hat{\mu}, \mu)^2 $$

However, if today we are talking about vectors, where $\boldsymbol{\hat{\mu}}$ is an estimator of $\boldsymbol{\mu}$, both of which are $n\times 1$ vectors, I was wondering if there is a corresponding nice decomposition as in the scalar case above for:

$$ E\left[||\boldsymbol{\hat{\mu}}-\boldsymbol{\mu}||^2\right] $$ ?

Thanks!

$\endgroup$

1 Answer 1

7
$\begingroup$

Simply note that

$$|| \widehat{\mu} - \mu ||^2 = \sum\limits_{i = 1}^{n} (\widehat{\mu}_{i} - \mu_{i})^2$$

Then, the answer is given by the decomposition you gave earlier:

$$ \mathbb{E}[(\widehat{\mu}_{i} - \mu_{i})^2] = Var[\widehat{\mu}_{i}] + [Bias(\widehat{\mu}_{i}, \mu_{i} )]^2 $$

Summing all up, we get

$$ \mathbb{E}[||\widehat{\mu} - \mu||^2] = \sum\limits_{i = 1}^{n} Var[\widehat{\mu}_{i}] + [Bias(\widehat{\mu}_{i}, \mu_{i} )]^2 $$

Another issue, totally different, is the covariance matrix $\mathbb{E}[(\widehat{\mu} - \mu)(\widehat{\mu} - \mu)^t]$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.