3
$\begingroup$

I'm trying to understand the concept of consistency in point estimation. Could somebody give me an illustrative example of a (simply) consistent estimator that is not MSE consistent?

Just a recall of definitions and theorems (I hope I have them right):

  • $\hat{\theta}_m$ is simply consistent estimator of parameter $\theta$ if $\forall \epsilon > 0$: $$\lim_{m \to \infty} \mathbb{P}[| \hat{\theta}_m - \theta | > \epsilon] = 0$$

  • $\hat{\theta}_m$ is MSE consistent estimator of parameter $\theta$ if: $$\lim_{m \to \infty} \mathbb{E}[(\hat{\theta}_m - \theta)^2] = 0$$

  • MSE can be decomposed into variance and bias: $$ \mathbb{E}[(\hat{\theta} - \theta)^2] = \underbrace{\mathbb{E}[(\hat{\theta} - \mathbb{E}[\hat{\theta}])^2]}_{\text{Var}[\hat{\theta}]} + \underbrace{(\mathbb{E}[\hat{\theta}] - \theta)^2}_{\text{Bias}_\theta[\hat{\theta}]^2} $$

  • MSE consistency implies simple consistency from Chebyshev's Inequality: $$\mathbb{P}[|\hat{\theta} - \theta| \geq \epsilon] \leq \frac{\mathbb{E}[(\hat{\theta} - \theta)^2]}{\epsilon^2}$$

$\endgroup$
  • 1
    $\begingroup$ The question is really about the difference between convergence in probability and in $L^2$. One can construct a sequence of r.v.'s (estimators are r.v.'s) that converge in probability but not $L^1$. $\endgroup$ – Michael Dec 15 '17 at 21:54
1
$\begingroup$

I think you are trying to say that convergence in probability does not imply $L^p$ convergence. see here

$\endgroup$
  • 1
    $\begingroup$ Agree, but an example of this in the context of an estimation problem would be interesting. $\endgroup$ – alexpghayes Aug 17 '19 at 20:30
1
$\begingroup$

Suppose $X_1, X_2, \cdots X_n \stackrel{iid}{\sim} N(\mu, 1)$ and our goal is to estimate $\mu$. Consider the estimator $$\hat\mu_n = \begin{cases} \bar X_n, & \text{with probability $\frac{n-1}{n}$} \\[1.2ex] n, & \text{with probability $\frac{1}{n}$} \end{cases}$$

By introducing $Y_n \sim Bern\left(\frac{n-1}{n}\right)$ with $Y_i \perp \!\!\! \perp Y_j$ (for all $i\neq j$) and $Y_i \perp \!\!\! \perp X_j$ (for all $i$ and $j$) we can rewrite this estimator as $$\hat\mu_n = \bar X_n Y_n + n(1-Y_n) .$$

This estimator is simply consistent.

To see this, consider \begin{align*} P(|\hat\mu_n - \mu| < \epsilon) &= P(|\bar X_n - \mu| < \epsilon)\frac{n-1}{n} + P(|n-\mu|<\epsilon)\frac{1}{n} \\[1.3ex] &=\frac{n-1}{n}\left\{\Phi\left(\sqrt n \epsilon \right)- \Phi\left(-\sqrt n \epsilon \right) \right\} + P(|n-\mu|<\epsilon)\frac{1}{n} \end{align*}

Taking the limit gives $$\lim_{n\rightarrow\infty} P(|\hat\mu_n - \mu| < \epsilon) = 1,$$ and thus $\hat\mu_n \stackrel{p}{\rightarrow} \mu$.

This estimator is not MSE-consistent

We start by finding the bias.

\begin{align*} E(\hat\mu_n) &= E(\bar X_n Y_n + n(1-Y_n)) \\[1.2ex] &= E(\bar X_n)E(Y_n) + nE(1-Y_n) \\[1.2ex] &= \mu\frac{n-1}{n} + n\frac{n-1}{n} \\[1.2ex] &= \frac{n-1}{n}\mu + 1 \end{align*}

Therefore $B_\mu(\hat\mu_n) = 1 - \mu/n$ which does not converge to $0$ as $n \rightarrow \infty$. Since $Var(\hat\mu_n)$ must be non-negative this is enough to conclude that $MSE(\hat\mu_n) \not\rightarrow 0$, and therefore the estimator is not MSE-consistent.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.