Consistent estimator, that is not MSE consistent

Question

I'm trying to understand the concept of consistency in point estimation. Could somebody give me an illustrative example of a (simply) consistent estimator that is not MSE consistent?

Just a recall of definitions and theorems (I hope I have them right):

• $\hat{\theta}_m$ is simply consistent estimator of parameter $\theta$ if $\forall \epsilon > 0$: $$\lim_{m \to \infty} \mathbb{P}[| \hat{\theta}_m - \theta | > \epsilon] = 0$$

• $\hat{\theta}_m$ is MSE consistent estimator of parameter $\theta$ if: $$\lim_{m \to \infty} \mathbb{E}[(\hat{\theta}_m - \theta)^2] = 0$$

• MSE can be decomposed into variance and bias: $$ \mathbb{E}[(\hat{\theta} - \theta)^2] = \underbrace{\mathbb{E}[(\hat{\theta} - \mathbb{E}[\hat{\theta}])^2]}_{\text{Var}[\hat{\theta}]} + \underbrace{(\mathbb{E}[\hat{\theta}] - \theta)^2}_{\text{Bias}_\theta[\hat{\theta}]^2} $$

• MSE consistency implies simple consistency from Chebyshev's Inequality: $$\mathbb{P}[|\hat{\theta} - \theta| \geq \epsilon] \leq \frac{\mathbb{E}[(\hat{\theta} - \theta)^2]}{\epsilon^2}$$

Answer 1

Suppose $X_1, X_2, \cdots X_n \stackrel{iid}{\sim} N(\mu, 1)$ and our goal is to estimate $\mu$. Consider the estimator $$\hat\mu_n = \begin{cases} \bar X_n, & \text{with probability $\frac{n-1}{n}$} \\[1.2ex] n, & \text{with probability $\frac{1}{n}$} \end{cases}$$

By introducing $Y_n \sim Bern\left(\frac{n-1}{n}\right)$ with $Y_i \perp \!\!\! \perp Y_j$ (for all $i\neq j$) and $Y_i \perp \!\!\! \perp X_j$ (for all $i$ and $j$) we can rewrite this estimator as $$\hat\mu_n = \bar X_n Y_n + n(1-Y_n) .$$

This estimator is simply consistent.

To see this, consider \begin{align*} P(|\hat\mu_n - \mu| < \epsilon) &= P(|\bar X_n - \mu| < \epsilon)\frac{n-1}{n} + P(|n-\mu|<\epsilon)\frac{1}{n} \\[1.3ex] &=\frac{n-1}{n}\left\{\Phi\left(\sqrt n \epsilon \right)- \Phi\left(-\sqrt n \epsilon \right) \right\} + P(|n-\mu|<\epsilon)\frac{1}{n} \end{align*}

Taking the limit gives $$\lim_{n\rightarrow\infty} P(|\hat\mu_n - \mu| < \epsilon) = 1,$$ and thus $\hat\mu_n \stackrel{p}{\rightarrow} \mu$.

This estimator is not MSE-consistent

We start by finding the bias.

\begin{align*} E(\hat\mu_n) &= E(\bar X_n Y_n + n(1-Y_n)) \\[1.2ex] &= E(\bar X_n)E(Y_n) + nE(1-Y_n) \\[1.2ex] &= \mu\frac{n-1}{n} + n\frac{1}{n} \\[1.2ex] &= \frac{n-1}{n}\mu + 1 \end{align*}

Therefore $B_\mu(\hat\mu_n) = 1 - \mu/n$ which does not converge to $0$ as $n \rightarrow \infty$. Since $Var(\hat\mu_n)$ must be non-negative this is enough to conclude that $MSE(\hat\mu_n) \not\rightarrow 0$, and therefore the estimator is not MSE-consistent.

Answer 2

I think you are trying to say that convergence in probability does not imply $L^p$ convergence. see here

Answer 3

A real-world example is the parameter estimates in logistic regression. For simplicity, assume a single predictor variable $X$. The model is $$\mathrm{logit}\,P(Y=1|X=x)=x\beta.$$

If all the observations with $Y=1$ have larger $X$ than all the observations with $Y=0$, $\hat\beta$ is infinite. The probability of this is exponentially small in $n$, but non-zero for all finite $n$.

Because the probability is exponentially small for large $n$, $\hat\beta\stackrel{p}{\to}\beta$. On the other hand, because the probability is not zero, $\hat\beta$ has no finite moments and so does not converge in mean square.