Does the MLE converge in mean-square?

Question

Simple question: does the MLE of a (finite-dimensional) parameter converge in mean square to the true value, that is, $$\mathbb E[\Vert\hat\theta_\text{MLE} - \theta\Vert^2_2]\rightarrow 0.$$ I know that, under certain conditions, the MLE is consistent, asymptotically normal, and asymptotically efficient. So I would say the answer to my question is negative.

However, consider the MLE of mean and variance of a normal distribution. It is given by $$\hat\theta_\text{MLE} = n^{-1}\sum_{i=1}^n\begin{bmatrix} X_i \\ \left(X_i - n^{-1}\sum_{i=1}^nX_i\right)^2\end{bmatrix}.$$ This MLE is asymptotically unbiased in the sense that $$\Vert\mathbb E[\hat\theta_\text{MLE} - \theta]\Vert_2\rightarrow 0,$$ and since the variance is given by $$\mathbb V[\hat\theta_\text{MLE}] = n^{-1}\begin{bmatrix}\sigma^2 & \mu_3 \\ \mu_3 & \frac{(n-1)}{n}\sigma^4\end{bmatrix},$$ it follows that $$\mathrm{trace}\big(\mathbb V[\hat\theta_\text{MLE}]\big)\rightarrow0.$$ Since $$\mathbb E[\Vert\hat\theta_\text{MLE} - \theta\Vert^2_2] = \mathrm{trace}\big(\mathbb V[\hat\theta_\text{MLE}]\big) + \Vert\mathbb E[\hat\theta_\text{MLE} - \theta]\Vert_2^2$$ the MLE converges in mean square. So does it indeed hold that the MLE converges in mean squre sense? If so, what are the conditions to establish mean square? And why is this not stated in textbooks?!

Answer 1

I think this paper answers your question! I will briefly describe the main result of the paper.

The paper suggests a family of distributions that are 'in between' the uniform and triangular distribution. That is, at $\theta = 1$ the distribution is the triangular distribution and for $\theta = 0$ the distribution is uniform. For in between values the distribution is something else.

The pdf is $$f(x \mid \theta) = \frac{(1-\theta)}{\delta(\theta)}\left(1 - \frac{|x-\theta|}{\delta(\theta)}\right)\times\mathbb{I}(x \in A) + \frac{\theta}{2}\mathbb{I}(x \in [-1,1])$$

Where $\mathbb{I}(\cdot)$ is the usual indicator function, A is the interval $[\theta - \delta(\theta), \theta + \delta(\theta)]$ and $\delta(\theta)$ is a function with the following properties

• $\delta(\theta)$ is continuous and decreasing in $\theta$
• $\delta(0) = 1$
• $0<\delta(\theta)<1-\theta$.

The main result:

Let $\hat{\theta}_n$ be the MLE for $\theta$ at a sample size of $n$. If $\delta(\theta) \to 0$ sufficiently fast as $n\to \infty$ then $\hat{\theta}_n \to 1$ with probability $1$, for any true $\theta \in [0,1]$.

Therefore, the MLE cannot converge in mean square to the true value!

---

An example of a function $\delta(\theta)$ which causes the MLE to ''break down'' is $$\delta(\theta) = \frac{\exp(-(1-\theta)^{-2})}{1-\theta} $$