3
$\begingroup$

Let $X_i$ be a random variable, and define the following statistic

$$ s^2_? = \frac{1}{n} \sum _i X_i ^2$$

  • Suppose that the $X_i$ comes from a symmetric distribution, is $s^2_?$ an unbiased estimator of the variance ($\sigma^2$) of $X_i$?

I tried the following:

$$E[s^2_?]=E[\frac{1}{n} \sum _i X_i ^2]=E[\frac{1}{n}\sum _i (X_i-\mu) ^2+2\mu X_i -\mu^2 ] $$

Considering a $\mu$ which satisfies $\mu= E[X_i]=0$

Then for the linearity of the expectation

$$E[s^2_?]=E[\frac{1}{n}\sum _i (X_i-\mu) ^2] + \frac{2\mu}{n}E[\sum_i X_i]+\frac{\mu^2}{n} E[\sum_i 1]$$

So

$$E[s^2_?]=\sigma^2+ \frac{2\mu}{n}E[\sum_i X_i]+\mu^2 E[\sum_i 1]$$

And, been that $\mu=0$

$$ s^2_? = \sigma^2$$


In case that the derivation is correct: How to adapt this to the case in which $X$ is autocorrelated?

Thanks in advance

$\endgroup$
3
$\begingroup$

The condition of symmetry is not required here. If you have known mean $\mu = 0$ then, following the working you used (but with standard notation), you get:

$$\begin{equation} \begin{aligned} \mathbb{E}(S_\mu^2) = \mathbb{E}\Big( \frac{1}{n} \sum_{i=1}^n X_i^2 \Big) &= \mathbb{E}\Big( \frac{1}{n} \sum_{i=1}^n (X_i - \mu)^2 \Big) \\[6pt] &= \frac{1}{n} \sum_{i=1}^n \mathbb{E}\Big((X_i - \mu)^2 \Big) \\[6pt] &= \frac{1}{n} \sum_{i=1}^n \mathbb{V}(X_i) \\[6pt] &= \frac{1}{n} \sum_{i=1}^n \sigma^2 \\[6pt] &= \frac{1}{n} \cdot n \sigma^2 = \sigma^2. \\[6pt] \end{aligned} \end{equation}$$

The above working is perfectly valid in the case where the random variables are correlated. (We can still write the expectation of the sum as the sum of the expectations, etc.) This is because the true mean is assumed known in this case, so correlation between the observable values has no effect on the expected value of the estimator. That means that this estimator is an unbiased estimator of the variance, for the case where $\mu = 0$, regardless of the correlation between the observable variables.

It is important to bear in mind that correlation between the values affects the variance of the estimator, and so confidence intervals are affected. There are other questions on this site that look at adjustments to the effective sample size (and consequent standard error of the estimator) for autocorrelation (see e.g., here). Roughly speaking, negative autocorrelation induces higher sample variance (relative to the true variance) in the series and positive autocorrelation induces lower sample variance (relative to the true variance). This necessitates adjustments to the standard error estimator, which can be calculated by deriving the variance of the estimator.

$\endgroup$
  • $\begingroup$ +1. Thank you. Could you comment about the case in which the random variable is autocorrelated? $\endgroup$ – user1420303 Mar 6 at 23:35
  • $\begingroup$ I have added additional discussion of correlation. $\endgroup$ – Ben Mar 7 at 0:19
3
$\begingroup$

I think it's a good idea to state it more clearly and simplify the solution a little bit.

Let $X_1,\dots,X_n$ be a random sample from a symmetric distribution with finite variance $\sigma^2$.

The variance estimator $$ \hat{\theta}=\frac{1}{n}\sum_{i=1}^n X_i^2 $$ is unbiased, because $$ \text{E}[\hat{\theta}] = \frac{1}{n} \sum_{i=1}^n \text{E}[X_i^2] = \frac{1}{n} \sum_{i=1}^n \left(\text{Var}[X_i] + \underbrace{\text{E}^2[X_i]}_{=\;0}\right)= \sigma^2. $$

$\endgroup$
  • $\begingroup$ +1. Thank you. Could you comment about the case in which the random variable is autocorrelated? $\endgroup$ – user1420303 Mar 6 at 23:35
1
$\begingroup$

Or, alternatively:

If $\mu=0$, then: $$\sigma^2 = E(X^2)+E(X)^2 = E(X^2)$$ and

$$E(\dfrac{1}{n} \sum_iX_i^2) = E(\bar{X^2}) = \bar{X^2} = E(X^2)$$

Then, by equating: $$\sigma^2 = E(\dfrac{1}{n} \sum_iX_i^2)$$

Thus, $s_?^2$ is an unbiased estimator of $\sigma^2$.

$\endgroup$
  • $\begingroup$ +1. Thank you. Could you comment about the case in which the random variable is autocorrelated? $\endgroup$ – user1420303 Mar 6 at 23:35
  • $\begingroup$ The last equation sign in this working is incorrect, and should be replaced with an approximation sign --- there is a difference between the second raw moment of a random variable and the corresponding sample estimator. In general, it is not true that $S_\mu^2 = \sigma^2.$ $\endgroup$ – Ben Mar 7 at 0:20
  • $\begingroup$ That's correct. I clarified my answer to avoid confusion. $\endgroup$ – Kuma Mar 8 at 13:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.