Does minimizing expected squared loss (MSE) result in an unbiased estimator? I have heard that the estimator with the lowest expected squared loss (mean squared error) is not always unbiased, but I have also heard that the constant that minimizes the expected squared loss vs. a random variable is the conditional mean. I believe the latter is unbiased. How do I reconcile these ideas?
 A: The answer to the question in your title is No. Here is a counterexample that may be of interest. It concerns two
possible estimators of the normal variance. Consider samples
of size $n = 5$ from $\mathsf{Norm}(\mu = 25, \sigma = 2),$
so that $\sigma^2 = 4.$
The usual estimate of $\sigma^2$ is $S^2 = \frac{\sum_{i=1}^n (X_i-\bar X)^2}{n-1} = Q/(n-1),$ which is unbiased: $E(S^2) = \sigma^2.$ This estimate is denoted
v in the R code below. For my choice of parameters $\text{MSE}(S^2) \approx 7.8.$ 
An alternative estimate of $\sigma^2$ (denoted v1 in the code)
might be $Q/(n+1),$ which is biased with $E(Q/(n+1)) \approx 2.7$, but has smaller mean-square error, $\text{MSE}(Q/(n+1)) \approx 5.3 < 7.8.$
Exact formulas are not difficult to derive analytically, but I show a simple
simulation in R that provides a good enough approximation to make the point. The long right tails of the distributions of the estimators heavily influence their MSEs.
set.seed(2019)
m = 10^6;  n = 5;  mu = 25;  sg = 2
v = replicate(m, var(rnorm(n,mu,sg)))
mean(v); mean((v - 4)^2)
[1] 3.997463   # aprx E(S^2) = 4
[1] 7.979501   # aprx MSE(S^2)
v1 = (n-1)*v/(n+1)
mean(v1); mean((v1 - 4)^2)
[1] 2.664975   # aprx E(Q/(n+1)) biased
[1] 5.328733   # aprx MSE(Q/(n+1)) < MSE(S^2)


Notes: A. A few respected statisticians have argued that one ought to use the estimator
$Q/n$ instead of $S^2$ because it has smaller MSE than $S^2$ and avoids the confusion of
explaining to elementary students about using $n - 1$ in the denominator.
Some substantial objections: (i) $Q/(n+1)$ has even smaller MSE. (ii) A change to $Q/n$ would
require major re-writing of accounts of estimation and testing
for $\sigma^2$ based on chi-squared distributions. (iii) Underestimates of
population variances may be undesirable in practical situations.
B. For larger values of $n,$ differences between the MSEs of the two
estimators become relatively unimportant, but the unbiased estimator never has the smaller MSE.
