2
$\begingroup$

Let's consider the Mean Square Error of an approximation of a parameter $\theta$ by $\hat{\theta}$.

$$\mathbb{E}(\theta-\hat{\theta})^2=Var(\hat{\theta})+(Bias(\hat{\theta}))^{2}$$

Usually, we say that there is a trade-off between the Variance and the Bias of the estimate, i.e. when the Bias is decreasing the Variance is increasing, and the opposite.

Is it possible to find an estimate that increases both the Variance and Bias?? In that case, we can say that the estimate is totally unworthy?

$\endgroup$

1 Answer 1

1
$\begingroup$

Absolutely! So when you say "increase", I assume you start from some baseline estimator. Say you have a i.i.d. sample $X_1, ..., X_n$ and want to find the mean $\theta = E[X]$. Your baseline estimator is the sample mean $\hat{\theta} = \frac{1}{n} \sum_{i=1}^n X_i$.
Recall that you can make any unbiased estimator biased by simply adding some constant $c \neq 0$. Now consider the estimator $\tilde{\theta} = X_1 + 1$. It should be clear that $\tilde{\theta}$ has both a larger variance than $\hat{\theta}$ and is also biased now.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.