# Does the MLE converge in mean-square?

Simple question: does the MLE of a (finite-dimensional) parameter converge in mean square to the true value, that is, $$\mathbb E[\Vert\hat\theta_\text{MLE} - \theta\Vert^2_2]\rightarrow 0.$$ I know that, under certain conditions, the MLE is consistent, asymptotically normal, and asymptotically efficient. So I would say the answer to my question is negative.

However, consider the MLE of mean and variance of a normal distribution. It is given by $$\hat\theta_\text{MLE} = n^{-1}\sum_{i=1}^n\begin{bmatrix} X_i \\ \left(X_i - n^{-1}\sum_{i=1}^nX_i\right)^2\end{bmatrix}.$$ This MLE is asymptotically unbiased in the sense that $$\Vert\mathbb E[\hat\theta_\text{MLE} - \theta]\Vert_2\rightarrow 0,$$ and since the variance is given by $$\mathbb V[\hat\theta_\text{MLE}] = n^{-1}\begin{bmatrix}\sigma^2 & \mu_3 \\ \mu_3 & \frac{(n-1)}{n}\sigma^4\end{bmatrix},$$ it follows that $$\mathrm{trace}\big(\mathbb V[\hat\theta_\text{MLE}]\big)\rightarrow0.$$ Since $$\mathbb E[\Vert\hat\theta_\text{MLE} - \theta\Vert^2_2] = \mathrm{trace}\big(\mathbb V[\hat\theta_\text{MLE}]\big) + \Vert\mathbb E[\hat\theta_\text{MLE} - \theta]\Vert_2^2$$ the MLE converges in mean square. So does it indeed hold that the MLE converges in mean squre sense? If so, what are the conditions to establish mean square? And why is this not stated in textbooks?!

• My guess is that there must be some counterexample, this cannot be true in full generality! Commented Feb 19, 2021 at 12:51
• that's my guess too. However, I couldn't find one. I suppose the counter example "pathological" and thus not easy to come up with one. My problem is that, if we insist that common regularity conditions hold (e.g. compact parameter domain, $\mathcal C^p$-ness of likelihood function, etc.) to ensure the above mentioned properties, it is very hard -- at least for me -- to construct such an example Commented Feb 19, 2021 at 13:15
• Maybe you could get larger interest for your post if you edited to focus on finding such a counterexample? Commented Feb 19, 2021 at 13:20
• I will start a bounty and hope that this will attract more attention. It may help already if someone can name sufficient conditions for mean square convergence Commented Feb 19, 2021 at 15:04
• The Neyman-Scott problem is such a counterexample where we have an inconsistent MLE stat.berkeley.edu/~census/neyscpar.pdf. You need to state some conditions for the problem to be interesting! Commented Feb 25, 2021 at 17:22

I think this paper answers your question! I will briefly describe the main result of the paper.

The paper suggests a family of distributions that are 'in between' the uniform and triangular distribution. That is, at $$\theta = 1$$ the distribution is the triangular distribution and for $$\theta = 0$$ the distribution is uniform. For in between values the distribution is something else.

The pdf is $$f(x \mid \theta) = \frac{(1-\theta)}{\delta(\theta)}\left(1 - \frac{|x-\theta|}{\delta(\theta)}\right)\times\mathbb{I}(x \in A) + \frac{\theta}{2}\mathbb{I}(x \in [-1,1])$$

Where $$\mathbb{I}(\cdot)$$ is the usual indicator function, A is the interval $$[\theta - \delta(\theta), \theta + \delta(\theta)]$$ and $$\delta(\theta)$$ is a function with the following properties

• $$\delta(\theta)$$ is continuous and decreasing in $$\theta$$
• $$\delta(0) = 1$$
• $$0<\delta(\theta)<1-\theta$$.

The main result:

Let $$\hat{\theta}_n$$ be the MLE for $$\theta$$ at a sample size of $$n$$. If $$\delta(\theta) \to 0$$ sufficiently fast as $$n\to \infty$$ then $$\hat{\theta}_n \to 1$$ with probability $$1$$, for any true $$\theta \in [0,1]$$.

Therefore, the MLE cannot converge in mean square to the true value!

An example of a function $$\delta(\theta)$$ which causes the MLE to ''break down'' is $$\delta(\theta) = \frac{\exp(-(1-\theta)^{-2})}{1-\theta}$$

An example of an MLE that converges in probability but not in mean square is the ratio of two binomials. Let $$X_n\sim Bin(n,p)$$ and $$Y_n\sim Bin(n,q)$$, then the MLE of $$(p,q)$$ is $$(\bar X_n, \bar Y_n)$$ and so the MLE of $$p/q$$ is $$\bar X_n/\bar Y_n$$. The MLE of $$(p,q)$$ is consistent by the LLN and the MLE of $$p/q$$ is consistent by the continuous mapping theorem, so we have convergence in probability (and almost surely).

However, with positive probability for any finite $$n$$, $$q$$ is zero so $$p/q$$ is infinite: it has no variance for any finite $$n$$ and does not converge in mean square.

Basically, the argument in the question depends on the variance of $$\hat\theta_n$$ converging to the variance of the asymptotic distribution, which does not happen in general.

If the parameter space is bounded the variance of $$\hat\theta_n$$ is uniformly bounded and convergence in probability does imply convergence in mean square