# Variance Bias Tradeoff

Let's consider the Mean Square Error of an approximation of a parameter $$\theta$$ by $$\hat{\theta}$$.

$$\mathbb{E}(\theta-\hat{\theta})^2=Var(\hat{\theta})+(Bias(\hat{\theta}))^{2}$$

Usually, we say that there is a trade-off between the Variance and the Bias of the estimate, i.e. when the Bias is decreasing the Variance is increasing, and the opposite.

Is it possible to find an estimate that increases both the Variance and Bias?? In that case, we can say that the estimate is totally unworthy?

## 1 Answer

Absolutely! So when you say "increase", I assume you start from some baseline estimator. Say you have a i.i.d. sample $$X_1, ..., X_n$$ and want to find the mean $$\theta = E[X]$$. Your baseline estimator is the sample mean $$\hat{\theta} = \frac{1}{n} \sum_{i=1}^n X_i$$.
Recall that you can make any unbiased estimator biased by simply adding some constant $$c \neq 0$$. Now consider the estimator $$\tilde{\theta} = X_1 + 1$$. It should be clear that $$\tilde{\theta}$$ has both a larger variance than $$\hat{\theta}$$ and is also biased now.