Are the MSEs of all estimators that have an MSE, equivalent asymptotically or are some estimators terribly bad even when sample size approaches infinity?
I was thinking about estimators with MSEs dependent on sample size and those have MSE of zero as limit as n approaches infinity.
There is no restriction on the term estimator, I can define an estimator for an unknown parameter $\theta$, as $\hat\theta=1$ and call it my estimator. It's a bad estimator that uses no data, and doesn't make MSE shrink towards zero. The MSE of an estimator is actually $MSE(\hat\theta)=E[(\hat\theta-\theta)^2]$, where it has variance and bias terms. These terms not necessarily approach zero.
Counterexample time!
Let's take a random sample $X_1,\dots,X_n \overset{iid}{\sim} N(\mu,1)$. As our first estimate of $\mu$, let's use $\hat{\mu}_1 = \bar{X}$. As an alternative estimator of $\mu$, lets use $\hat{\mu}_2 = 2$. Despite this looking silly, this is an admissible estimator in the sense that it has lower $MSE$ than $\hat{\mu}_1$ when the truth is that $\mu=2$.
We can decompose $MSE$ into the bias and variance: $MSE = bias^2 + variance$. Let's look at the $MSE$ of each estimator.
$MSE_1 = 0^2 + \frac{1}{n}$. As $n \rightarrow 0$, $MSE_1 \rightarrow 0$.
$MSE_2 = (\mu-2)^2 + 0$. As $n \rightarrow 0$, $MSE_2 \rightarrow \mu^2 - 4\mu + 4$.
No, these are not asymptotically equivalent.
