how to prove that one mean estimator converges faster than another Say that I have a target normal distribution $N=(\mu,\sigma)$. I have two estimators for $\mu$, say $\mu_1$ and $\mu_2$ that are estimated iteratively from $t=1...T$. Between $\mu_1$ and $\mu_2$, I know that $\mu_1$ has less bias than $\mu_2$ in estimating the true mean $\mu$ and is therefore the better estimator. How should I express in a formal way that using $\mu_1$ allows the estimated distribution $N_1$ to converge faster to $N$ as I iterate up to $T$?
 A: @AdamO points out, quite rightly that an estimator "is not necessarily the better estimator because it has less bias. An estimator with more bias can in fact converge much faster." For instance, one might ask "How do we measure the rate of convergence for an estimator" and secondarily "How can we compare two possibly biased estimators in terms of their rate of convergence?"
A normal distribution is not an especially good example of what you are trying to ask.
The best measure of location for a distribution is its minimum variance unbiased estimator MVUE. Let us take estimation of parameters of the Cauchy distribution. The mean is not robust, in fact, it will diverge from the Cauchy distribution's location parameter when more data is added because of the heavy tails of that distribution. The median is robust and converges to the distribution's location. A properly trimmed mean will be more robust than the median, and converge to the distribution's location faster.
Another exemplary distribution is the uniform distribution, the median will converge to the location, the mean will converge faster, and the average extreme value will converge to the uniform distribution's location faster still.
How one finds which measurement converges faster to the location is to look it up. For one thing, it takes mental effort coupled with trial and error to come up with better measurements if one starts from scratch. One can confirm easily enough, once one has an answer, what works better, but, one cannot specify a general methodology for creating better measurements other than to say, it is a bit like asking how one performs magic. Intuition perhaps, after having seen it done for a number of different cases. For example, a Pareto type I distribution has harmonic mean as a best measure of location overall, but demonstrating that might require doing Monte Carlo simulations.
Finally, robustness is sometimes more important than bias, for example, ridge regression introduces bias which can decrease variance and increase robustness at the cost of that bias, nor is bias for fitting necessarily inappropriate as ill-posed problems may have optimal solutions that are not optimal for the data fitting, but rather are optimized for other considerations.
A: In frequentist analysis, an idea like an estimator being better than another leads to very complicated questions and there is no clear consensual answer to it. For example, people often say being unbiased is "desirable", but this does not make an estimator necessarily better. See wikipedia. A simple measure of estimator quality is the MSE $E((\hat\theta-\theta)^2)$. Note that there is never an estimator with universally best MSE (among all estimators). The unbiased assumption is sometimes used just because there may be an estimator with universally best MSE among unbiased estimators. But unbiased does not mean better.
Let's assume $\sigma$ is known.

How should I express in a formal way that using $\mu_1$ allows the
  estimated distribution $N_1$ to converge faster to $N$ as I iterate up to
  T?

Well, first formalize this. Two normal distributions (with same variance) are close when their means are close. You could use a formal definition of a distance between distributions, but in this simple case, we can say you're interested in $|\mu-\mu_1|$. Hopefully this tends towards 0.
Since $\mu_1$ is a random variable, we need a convergence measure for random variables. For example $L^2$: we look at $d_1=E((\mu-\mu_1) ^2)$. Note that at this point, there are other ways to express the limit of random variables, but this one may be the simplest for a first study.
Finally, you can try to prove $d_1$ converges faster towards 0 than $d_2$ which is done by studying the ratio $d_1/d_2$, either with formal proofs or simulations. But fundamentally, this may depend on $\mu$, which makes the frequentist analysis tricky. What you can reasonably hope is to find a result that is independent of $\mu$. This often happens with normality.
