Does the MLE converge in mean-square?

Question

Simple question: does the MLE of a (finite-dimensional) parameter converge in mean square to the true value, that is, $$\mathbb E[\Vert\hat\theta_\text{MLE} - \theta\Vert^2_2]\rightarrow 0.$$ I know that, under certain conditions, the MLE is consistent, asymptotically normal, and asymptotically efficient. So I would say the answer to my question is negative.

However, consider the MLE of mean and variance of a normal distribution. It is given by $$\hat\theta_\text{MLE} = n^{-1}\sum_{i=1}^n\begin{bmatrix} X_i \\ \left(X_i - n^{-1}\sum_{i=1}^nX_i\right)^2\end{bmatrix}.$$ This MLE is asymptotically unbiased in the sense that $$\Vert\mathbb E[\hat\theta_\text{MLE} - \theta]\Vert_2\rightarrow 0,$$ and since the variance is given by $$\mathbb V[\hat\theta_\text{MLE}] = n^{-1}\begin{bmatrix}\sigma^2 & \mu_3 \\ \mu_3 & \frac{(n-1)}{n}\sigma^4\end{bmatrix},$$ it follows that $$\mathrm{trace}\big(\mathbb V[\hat\theta_\text{MLE}]\big)\rightarrow0.$$ Since $$\mathbb E[\Vert\hat\theta_\text{MLE} - \theta\Vert^2_2] = \mathrm{trace}\big(\mathbb V[\hat\theta_\text{MLE}]\big) + \Vert\mathbb E[\hat\theta_\text{MLE} - \theta]\Vert_2^2$$ the MLE converges in mean square. So does it indeed hold that the MLE converges in mean squre sense? If so, what are the conditions to establish mean square? And why is this not stated in textbooks?!

Answer 1

I think this paper answers your question! I will briefly describe the main result of the paper.

The paper suggests a family of distributions that are 'in between' the uniform and triangular distribution. That is, at $\theta = 1$ the distribution is the triangular distribution and for $\theta = 0$ the distribution is uniform. For in between values the distribution is something else.

The pdf is $$f(x \mid \theta) = \frac{(1-\theta)}{\delta(\theta)}\left(1 - \frac{|x-\theta|}{\delta(\theta)}\right)\times\mathbb{I}(x \in A) + \frac{\theta}{2}\mathbb{I}(x \in [-1,1])$$

Where $\mathbb{I}(\cdot)$ is the usual indicator function, A is the interval $[\theta - \delta(\theta), \theta + \delta(\theta)]$ and $\delta(\theta)$ is a function with the following properties

• $\delta(\theta)$ is continuous and decreasing in $\theta$
• $\delta(0) = 1$
• $0<\delta(\theta)<1-\theta$.

The main result:

Let $\hat{\theta}_n$ be the MLE for $\theta$ at a sample size of $n$. If $\delta(\theta) \to 0$ sufficiently fast as $n\to \infty$ then $\hat{\theta}_n \to 1$ with probability $1$, for any true $\theta \in [0,1]$.

Therefore, the MLE cannot converge in mean square to the true value!

---

An example of a function $\delta(\theta)$ which causes the MLE to ''break down'' is $$\delta(\theta) = \frac{\exp(-(1-\theta)^{-2})}{1-\theta} $$

Answer 2

An example of an MLE that converges in probability but not in mean square is the ratio of two binomials. Let $X_n\sim Bin(n,p)$ and $Y_n\sim Bin(n,q)$, then the MLE of $(p,q)$ is $(\bar X_n, \bar Y_n)$ and so the MLE of $p/q$ is $\bar X_n/\bar Y_n$. The MLE of $(p,q)$ is consistent by the LLN and the MLE of $p/q$ is consistent by the continuous mapping theorem, so we have convergence in probability (and almost surely).

However, with positive probability for any finite $n$, $q$ is zero so $p/q$ is infinite: it has no variance for any finite $n$ and does not converge in mean square.

Basically, the argument in the question depends on the variance of $\hat\theta_n$ converging to the variance of the asymptotic distribution, which does not happen in general.

If the parameter space is bounded the variance of $\hat\theta_n$ is uniformly bounded and convergence in probability does imply convergence in mean square