2
$\begingroup$

I am reading a proof of the consistency of sample variance estimator.

http://webpages.cs.luc.edu/~jdg/w3teaching/stat_305/sp11/consistentsamplevariance.pdf

On page two, it argues that, since $A=\frac{1}{n}\sum_{i=1}^nX_i^2$ is an unbiased estimator of $\Bbb E(X^2)$ and since $Var(A)\to 0$ as $n\to\infty$, the estimator converges in probability to $\Bbb E(X^2)$, or $A\overset{\Bbb P}{\to}\Bbb E(X^2)$.

I am new to these convergence concepts, so how does this follow? Is it true that $$Unbiased + Vanishing\text{ }Variance \implies Convergence\text{ }in\text{ } Probability$$ holds in general?

$\endgroup$

1 Answer 1

3
$\begingroup$

Unbiasedness + vanishing variance implies convergence in mean-square; convergence in mean-square implies convergence in probability.

If $A$ is an unbiased estimator of some parameter $\alpha$ then (by definition) we have $\mathbb{E}(A) = \alpha$. Then, using Chebyshev's inequality we have:

$$\mathbb{P}(|A - \alpha| > \epsilon) = \mathbb{P}(|A - \mathbb{E}(A)| > \epsilon) \leqslant \frac{\mathbb{V}(A)}{\epsilon^2}.$$

If $\lim_{n \rightarrow \infty} \mathbb{V}(A) = 0$ then $\lim_{n \rightarrow \infty} \mathbb{P}(|A - \alpha| > \epsilon) = 0$ for all $\epsilon > 0$, which means $A \xrightarrow{P} \alpha$.

In your particular question, you have $\alpha = \mathbb{E}(X^2)$, so the result you give is correct. It is a specific case of a more general rule that unbiasedness and vanishing variance implies convergence in mean-square, which then implies convergence in probability.

$\endgroup$
2
  • 1
    $\begingroup$ Thank you so much for your input, @Ben. This is really helpful. But did you mean to say $\Bbb E(A)$ rather than $\Bbb E(a)$ in the equation? $\endgroup$ Jan 29, 2018 at 4:32
  • $\begingroup$ Yes I did - edited. Thanks for pointing that out. $\endgroup$
    – Ben
    Jan 29, 2018 at 4:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.