Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
Let's consider the Mean Square Error of an approximation of a parameter $\theta$ by $\hat{\theta}$.
$$\mathbb{E}(\theta-\hat{\theta})^2=Var(\hat{\theta})+(Bias(\hat{\theta}))^{2}$$
Usually, we say that there is a trade-off between the Variance and the Bias of the estimate, i.e. when the Bias is decreasing the Variance is increasing, and the opposite.
Is it possible to find an estimate that increases both the Variance and Bias?? In that case, we can say that the estimate is totally unworthy?
Absolutely! So when you say "increase", I assume you start from some baseline estimator. Say you have a i.i.d. sample $X_1, ..., X_n$ and want to find the mean $\theta = E[X]$. Your baseline estimator is the sample mean $\hat{\theta} = \frac{1}{n} \sum_{i=1}^n X_i$.
Recall that you can make any unbiased estimator biased by simply adding some constant $c \neq 0$.
Now consider the estimator $\tilde{\theta} = X_1 + 1$.
It should be clear that $\tilde{\theta}$ has both a larger variance than $\hat{\theta}$ and is also biased now.
