1
$\begingroup$

There are many ways to check if a sequence of estimators is consistent.

By definition, a sequence of estimators $W_n = W_n(X_1,X_2,\ldots,X_n)$ is a consistent sequence of estimators of the parameter $\theta$ if for every $\epsilon > 0$ $$\lim_{n\to\infty}\mathbf P_\theta (|W_n-\theta| < \epsilon) = 1$$

While it is not necessary that $\lim_{n\to\infty} E_\theta[(W_n-\theta)^2] = 0$, we can be sure that $W_n$ is a consistent sequence of estimators if $\lim_{n\to\infty} E_\theta[(W_n-\theta)^2] = 0$. This is due to Chebyshev's inequality: $$0 \le \mathbf P_\theta (|W_n-\theta| \ge \epsilon) \le \frac{E_\theta[(W_n-\theta)^2]}{\epsilon^2}$$ If $E_\theta[(W_n-\theta)^2] \to 0$ as $n\to\infty$, $\mathbf P_\theta (|W_n-\theta| \ge \epsilon) \to 0$ as $n\to \infty$, and we obtain $\lim_{n\to\infty}\mathbf P_\theta (|W_n-\theta| < \epsilon) = 1$.

Question: Since (apparently) $\lim_{n\to\infty} E_\theta[(W_n-\theta)^2] = 0$ is only a sufficient condition for $W_n$ to be a consistent sequence of estimators, and not a necessary one, how do I produce a consistent sequence of estimators $W_n$ for which $\lim_{n\to\infty} E_\theta[(W_n-\theta)^2]\ne 0$? If we cannot do this, the condition must be necessary and sufficient, which is stronger. Would appreciate any help, thank you!

$\endgroup$
2
$\begingroup$

Maximum likelihood estimators in logistic regression are one example. The MLE is infinite with non-zero probability, which stays non-zero for all $n$, though it decreases exponentially with $n$. $E[(W_n-\theta)^2]$ is infinite, and does not converge to zero.

If you don't like estimators that can be non-finite, take $W_n$ as mean of a sample of size $n$ from a $t_2$ distribution. The distribution has a well-defined mean, zero, but no finite variance, so $E[(W_n-\theta)^2]=E[W_n^2]$ is always infinite.

I don't know of any naturally occurring examples where $E[(W_n-\theta)^2]$ is finite but doesn't converge to zero, but it's easy to rig up examples if $W_n$ is allowed to be silly. For example, if you have $n$ iid observations from $N(\mu,1)$ and take $W_n=\bar X_n$ with probability $1-1/n$ and $W_n=n^{2/3}$ with probability $1/n$, then $W_n$ is consistent for $\mu$ (because $P(W_n=\bar X_n)\to 1$) but the variance doesn't vanish.

Basically, what you need is that a vanishing fraction of 'wrong' values of $W_n$ can be very very wrong, but most of the values of $W_n$ are close to $\theta$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.