I'm trying to understand the concept of consistency in point estimation. Could somebody give me an illustrative example of a (simply) consistent estimator that is not MSE consistent?

Just a recall of definitions and theorems (I hope I have them right):

  • $\hat{\theta}_m$ is simply consistent estimator of parameter $\theta$ if $\forall \epsilon > 0$: $$\lim_{m \to \infty} \mathbb{P}[| \hat{\theta}_m - \theta | > \epsilon] = 0$$

  • $\hat{\theta}_m$ is MSE consistent estimator of parameter $\theta$ if: $$\lim_{m \to \infty} \mathbb{E}[(\hat{\theta}_m - \theta)^2] = 0$$

  • MSE can be decomposed into variance and bias: $$ \mathbb{E}[(\hat{\theta} - \theta)^2] = \underbrace{\mathbb{E}[(\hat{\theta} - \mathbb{E}[\hat{\theta}])^2]}_{\text{Var}[\hat{\theta}]} + \underbrace{(\mathbb{E}[\hat{\theta}] - \theta)^2}_{\text{Bias}_\theta[\hat{\theta}]^2} $$

  • MSE consistency implies simple consistency from Chebyshev's Inequality: $$\mathbb{P}[|\hat{\theta} - \theta| \geq \epsilon] \leq \frac{\mathbb{E}[(\hat{\theta} - \theta)^2]}{\epsilon^2}$$

  • 1
    $\begingroup$ The question is really about the difference between convergence in probability and in $L^2$. One can construct a sequence of r.v.'s (estimators are r.v.'s) that converge in probability but not $L^1$. $\endgroup$
    – Michael
    Commented Dec 15, 2017 at 21:54

3 Answers 3


Suppose $X_1, X_2, \cdots X_n \stackrel{iid}{\sim} N(\mu, 1)$ and our goal is to estimate $\mu$. Consider the estimator $$\hat\mu_n = \begin{cases} \bar X_n, & \text{with probability $\frac{n-1}{n}$} \\[1.2ex] n, & \text{with probability $\frac{1}{n}$} \end{cases}$$

By introducing $Y_n \sim Bern\left(\frac{n-1}{n}\right)$ with $Y_i \perp \!\!\! \perp Y_j$ (for all $i\neq j$) and $Y_i \perp \!\!\! \perp X_j$ (for all $i$ and $j$) we can rewrite this estimator as $$\hat\mu_n = \bar X_n Y_n + n(1-Y_n) .$$

This estimator is simply consistent.

To see this, consider \begin{align*} P(|\hat\mu_n - \mu| < \epsilon) &= P(|\bar X_n - \mu| < \epsilon)\frac{n-1}{n} + P(|n-\mu|<\epsilon)\frac{1}{n} \\[1.3ex] &=\frac{n-1}{n}\left\{\Phi\left(\sqrt n \epsilon \right)- \Phi\left(-\sqrt n \epsilon \right) \right\} + P(|n-\mu|<\epsilon)\frac{1}{n} \end{align*}

Taking the limit gives $$\lim_{n\rightarrow\infty} P(|\hat\mu_n - \mu| < \epsilon) = 1,$$ and thus $\hat\mu_n \stackrel{p}{\rightarrow} \mu$.

This estimator is not MSE-consistent

We start by finding the bias.

\begin{align*} E(\hat\mu_n) &= E(\bar X_n Y_n + n(1-Y_n)) \\[1.2ex] &= E(\bar X_n)E(Y_n) + nE(1-Y_n) \\[1.2ex] &= \mu\frac{n-1}{n} + n\frac{1}{n} \\[1.2ex] &= \frac{n-1}{n}\mu + 1 \end{align*}

Therefore $B_\mu(\hat\mu_n) = 1 - \mu/n$ which does not converge to $0$ as $n \rightarrow \infty$. Since $Var(\hat\mu_n)$ must be non-negative this is enough to conclude that $MSE(\hat\mu_n) \not\rightarrow 0$, and therefore the estimator is not MSE-consistent.

  • $\begingroup$ There is a typo in your answer. In the penultimate step, the second term on the RHS should be $n \frac{1}{n}$ $\endgroup$
    – user67724
    Commented Mar 23, 2023 at 23:03

I think you are trying to say that convergence in probability does not imply $L^p$ convergence. see here

  • 1
    $\begingroup$ Agree, but an example of this in the context of an estimation problem would be interesting. $\endgroup$ Commented Aug 17, 2019 at 20:30

A real-world example is the parameter estimates in logistic regression. For simplicity, assume a single predictor variable $X$. The model is $$\mathrm{logit}\,P(Y=1|X=x)=x\beta.$$

If all the observations with $Y=1$ have larger $X$ than all the observations with $Y=0$, $\hat\beta$ is infinite. The probability of this is exponentially small in $n$, but non-zero for all finite $n$.

Because the probability is exponentially small for large $n$, $\hat\beta\stackrel{p}{\to}\beta$. On the other hand, because the probability is not zero, $\hat\beta$ has no finite moments and so does not converge in mean square.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.