Problems occur when a model $f(x,\theta)$ has a high tendency to fit the noise.
In that case the model tends to over-fit. That is, it is not only expressing the true model but also the random noise that you do not want to capture with your model (because the noise is a non-systematic part that does not allow you to make predictions for new data).
One might improve (reduce) the total error of fitting, by introducing some bias, when the this bias makes the variance/over-fitting reduce more strongly than the increase of the bias/under-fitting (ie not correctly representing the true model).
1. Why exactly $E[(\hat{\theta}_n - E[\hat{\theta}_n])^2]$ and $E[\hat{\theta}_n - \theta]$ cannot be decreased simultaneously?
This is not true. They can be decreased simultaneously (depending on the case). Imagine that you introduced some bias which both increased the variance as well as the bias. Then in the reverse direction reducing this bias will simultaneously reduce bias and variance.
For example a scaled root mean squared difference $c \sqrt{\frac{1}{n} {\sum(x_i-\bar{x})^2}}$ for sample of size $n$ is an unbiased estimator for the population standard deviation $\sigma$ when $c=\sqrt{\frac{n}{n-1}}$. Now, if you would have $c>\sqrt{\frac{n}{n-1}}$, then you would both reduce the bias as well as the variance when you reduce the size of this constant $c$.
However, the bias that is (intentionally) added in regularization is often of the kind that reduces the variance (e.g. you could reduce $c$ to a level below $\sqrt{\frac{n}{n-1}}$). Thus, in practice, you get a trade-off in bias versus variance and reducing the bias will increase the variance (and vice versa).
2. Why can't we just take some unbiased estimator and reduce the variance by increasing sample size?
In principle you can.
But,
- This may require much more sampling effort which is expensive, and this is often a limitation.
- Possibly there might also be computational difficulties with certain
estimation problems and the sample size would need to
increase extremely in order to solve this, if it is possible at all.
(e.g. high dimensionality parameters>measurements, or as in ridge regression: very shallow paths around the global optimum)
Often there is also no objection to bias. When it is about reducing the total error (as in many cases) then the use of a biased but less erroneous estimator is to be preferred.
About your counter example.
Related to your second question you can indeed reduce the error by increasing the sample size. And related to your first question you can also reduce both bias and variance (say you use a scaled sample mean $c\frac{\sum{x_i}}{n}$ as estimator of the population mean and consider varying the scaling parameter $c$).
However the region of practical interest is where the decreasing bias coincides with an increasing variance. The image below shows this contrast by using a sample (size = 5) taken from a normal distribution with variance = 1 and mean = 1. The unscaled sample mean is the unbiased predictor of the population mean. If you would increase the scaling of this predictor than you have both increasing bias and increasing variance. However if you decrease the scaling of the predictor then you have increasing bias, but decreasing variance. The "optimal" predictor is then actually not the sample mean but some shrinked estimator (see also Why is the James-Stein estimator called a "shrinkage" estimator?).
