# Consistent estimator, that is not MSE consistent

I'm trying to understand the concept of consistency in point estimation. Could somebody give me an illustrative example of a (simply) consistent estimator that is not MSE consistent?

Just a recall of definitions and theorems (I hope I have them right):

• $\hat{\theta}_m$ is simply consistent estimator of parameter $\theta$ if $\forall \epsilon > 0$: $$\lim_{m \to \infty} \mathbb{P}[| \hat{\theta}_m - \theta | > \epsilon] = 0$$

• $\hat{\theta}_m$ is MSE consistent estimator of parameter $\theta$ if: $$\lim_{m \to \infty} \mathbb{E}[(\hat{\theta}_m - \theta)^2] = 0$$

• MSE can be decomposed into variance and bias: $$\mathbb{E}[(\hat{\theta} - \theta)^2] = \underbrace{\mathbb{E}[(\hat{\theta} - \mathbb{E}[\hat{\theta}])^2]}_{\text{Var}[\hat{\theta}]} + \underbrace{(\mathbb{E}[\hat{\theta}] - \theta)^2}_{\text{Bias}_\theta[\hat{\theta}]^2}$$

• MSE consistency implies simple consistency from Chebyshev's Inequality: $$\mathbb{P}[|\hat{\theta} - \theta| \geq \epsilon] \leq \frac{\mathbb{E}[(\hat{\theta} - \theta)^2]}{\epsilon^2}$$

• The question is really about the difference between convergence in probability and in $L^2$. One can construct a sequence of r.v.'s (estimators are r.v.'s) that converge in probability but not $L^1$. Commented Dec 15, 2017 at 21:54

Suppose $$X_1, X_2, \cdots X_n \stackrel{iid}{\sim} N(\mu, 1)$$ and our goal is to estimate $$\mu$$. Consider the estimator $$\hat\mu_n = \begin{cases} \bar X_n, & \text{with probability \frac{n-1}{n}} \\[1.2ex] n, & \text{with probability \frac{1}{n}} \end{cases}$$

By introducing $$Y_n \sim Bern\left(\frac{n-1}{n}\right)$$ with $$Y_i \perp \!\!\! \perp Y_j$$ (for all $$i\neq j$$) and $$Y_i \perp \!\!\! \perp X_j$$ (for all $$i$$ and $$j$$) we can rewrite this estimator as $$\hat\mu_n = \bar X_n Y_n + n(1-Y_n) .$$

### This estimator is simply consistent.

To see this, consider \begin{align*} P(|\hat\mu_n - \mu| < \epsilon) &= P(|\bar X_n - \mu| < \epsilon)\frac{n-1}{n} + P(|n-\mu|<\epsilon)\frac{1}{n} \\[1.3ex] &=\frac{n-1}{n}\left\{\Phi\left(\sqrt n \epsilon \right)- \Phi\left(-\sqrt n \epsilon \right) \right\} + P(|n-\mu|<\epsilon)\frac{1}{n} \end{align*}

Taking the limit gives $$\lim_{n\rightarrow\infty} P(|\hat\mu_n - \mu| < \epsilon) = 1,$$ and thus $$\hat\mu_n \stackrel{p}{\rightarrow} \mu$$.

### This estimator is not MSE-consistent

We start by finding the bias.

\begin{align*} E(\hat\mu_n) &= E(\bar X_n Y_n + n(1-Y_n)) \\[1.2ex] &= E(\bar X_n)E(Y_n) + nE(1-Y_n) \\[1.2ex] &= \mu\frac{n-1}{n} + n\frac{1}{n} \\[1.2ex] &= \frac{n-1}{n}\mu + 1 \end{align*}

Therefore $$B_\mu(\hat\mu_n) = 1 - \mu/n$$ which does not converge to $$0$$ as $$n \rightarrow \infty$$. Since $$Var(\hat\mu_n)$$ must be non-negative this is enough to conclude that $$MSE(\hat\mu_n) \not\rightarrow 0$$, and therefore the estimator is not MSE-consistent.

• There is a typo in your answer. In the penultimate step, the second term on the RHS should be $n \frac{1}{n}$ Commented Mar 23, 2023 at 23:03

I think you are trying to say that convergence in probability does not imply $L^p$ convergence. see here

• Agree, but an example of this in the context of an estimation problem would be interesting. Commented Aug 17, 2019 at 20:30

A real-world example is the parameter estimates in logistic regression. For simplicity, assume a single predictor variable $$X$$. The model is $$\mathrm{logit}\,P(Y=1|X=x)=x\beta.$$

If all the observations with $$Y=1$$ have larger $$X$$ than all the observations with $$Y=0$$, $$\hat\beta$$ is infinite. The probability of this is exponentially small in $$n$$, but non-zero for all finite $$n$$.

Because the probability is exponentially small for large $$n$$, $$\hat\beta\stackrel{p}{\to}\beta$$. On the other hand, because the probability is not zero, $$\hat\beta$$ has no finite moments and so does not converge in mean square.