7
$\begingroup$

I am trying to prove that $s^2=\frac{1}{n-1}\sum^{n}_{i=1}(X_i-\bar{X})^2$ is a consistent estimator of $\sigma^2$ (variance), meaning that as the sample size $n$ approaches $\infty$ , $\text{var}(s^2)$ approaches 0 and it is unbiased.

I understand how to prove that it is unbiased, but I cannot think of a way to prove that $\text{var}(s^2)$ has a denominator of n. Does anyone have any ways to prove this?


The question asks:

A random sample of size n is taken from a normal population with variance $\sigma^2$. Show that the statistic $s^2$ is a consistent estimator of $\sigma^2$

So far I have gotten:
$\text{var}(s^2) = \text{var}(\frac{1}{n-1}\Sigma X^2-n\bar X^2)$
$= \frac{1}{(n-1)^2}(\text{var}(\Sigma X^2) + \text{var}(n\bar X^2))$
$= \frac{n^2}{(n-1)^2}(\text{var}(X^2) + \text{var}(\bar X^2))$

But as I do not know how to find $Var(X^2) $and$ Var(\bar X^2)$, I am stuck here (I have already proved that $S^2$ is an unbiased estimator of $Var(\sigma^2)$)

Source : Edexcel AS and A Level Modular Mathematics S4 (from 2008 syllabus) Examination Style Paper Question 1. This is for my own studies and not school work.

$\endgroup$
7
  • 1
    $\begingroup$ Please add the [self-study] tag & read its wiki. Also, you can use $\LaTeX$ markup on this site, see math.meta.stackexchange.com/questions/5020/…. $\endgroup$ – kjetil b halvorsen Nov 14 '20 at 9:51
  • 1
    $\begingroup$ The decomposition of the variance is incorrect in several aspects. $\endgroup$ – Xi'an Nov 14 '20 at 10:55
  • $\begingroup$ @Xi'an On the third line of working, I realised I did not put a ^2 on the n on the numerator of the fraction $\endgroup$ – MrDerpinati Nov 14 '20 at 11:02
  • $\begingroup$ @MrDerpinati, please have a look at my answer, and let me know if it's understandable to you or not. $\endgroup$ – Kolmogorov Nov 14 '20 at 11:47
  • 1
    $\begingroup$ @Xi'an My textbook did not cover the variation of random variables that are not independent, so I am guessing that if $X_i$ and $\bar X_n$ are dependent, $Var(X_i +\bar X_n) = Var(X_i) + Var(\bar X_n)$ ? $\endgroup$ – MrDerpinati Nov 14 '20 at 14:18
9
$\begingroup$

It's a very well known result that :

If $X_1, X_2, \cdots, X_n \stackrel{\text{iid}}{\sim} N(\mu,\sigma^2)$ , then $$Z_n = \dfrac{\displaystyle\sum(X_i - \bar{X})^2}{\sigma^2} \sim \chi^2_{n-1}$$ Thus, $ \mathbb{E}(Z_n) = n-1 $ and $ \text{var}(Z_n) = 2(n-1)$ .

If you wish to see a proof of the above result, please refer to this link.

Now, since you already know that $s^2$ is an unbiased estimator of $\sigma^2$ , so for any $\varepsilon>0$ , we have :

\begin{align*} &\mathbb{P}(\mid s^2 - \sigma^2 \mid > \varepsilon )\\ &= \mathbb{P}(\mid s^2 - \mathbb{E}(s^2) \mid > \varepsilon )\\ &\leqslant \dfrac{\text{var}(s^2)}{\varepsilon^2}\\ &=\dfrac{1}{(n-1)^2}\cdot \text{var}\left[\sum (X_i - \overline{X})^2)\right]\\ &=\dfrac{\sigma^4}{(n-1)^2}\cdot \text{var}\left[\frac{\sum (X_i - \overline{X})^2}{\sigma^2}\right]\\ &=\dfrac{\sigma^4}{(n-1)^2}\cdot\text{var}(Z_n)\\ &=\dfrac{\sigma^4}{(n-1)^2}\cdot 2(n-1) = \dfrac{2\sigma^4}{n-1} \stackrel{n\to\infty}{\longrightarrow} 0 \end{align*}

Thus, $ \displaystyle\lim_{n\to\infty} \mathbb{P}(\mid s^2 - \sigma^2 \mid > \varepsilon ) = 0$ , i.e. $ s^2 \stackrel{\mathbb{P}}{\longrightarrow} \sigma^2 $ as $n\to\infty$ , which tells us that $s^2$ is a consistent estimator of $\sigma^2$ .


Note : I have used Chebyshev's inequality in the first inequality step used above. Hope my answer serves your purpose. Thank you.

$\endgroup$
3
  • $\begingroup$ Since the OP is unable to compute the variance of $Z_n$, it is neither well-know nor straightforward for them. I thus suggest you also provide the derivation of this variance. $\endgroup$ – Xi'an Nov 14 '20 at 12:15
  • $\begingroup$ I feel like I have seen a similar answer somewhere before in my textbook (I couldn't find where!) but the method is very different. As I am doing 11th/12th grade (A Level in the UK) maths, to me, this seems like a university level answer, and thus I do not really understand this. Thank you for your input, but I am sorry to say I do not understand. $\endgroup$ – MrDerpinati Nov 14 '20 at 14:07
  • 1
    $\begingroup$ I guess there isn't any easier explanation to your query other than what I wrote. Also, what @Xi'an is talking about surely needs a proof which isn't very elementary (I've mentioned a link). In fact, the definition of Consistent estimators is based on Convergence in Probability. Do you know what that means ? $\endgroup$ – Kolmogorov Nov 14 '20 at 19:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.