0
$\begingroup$

I am trying to find the formal way of writing the sample variance of a continuous random variable considering Bessel's correction.

I ask because the sample variance is usually written this way:

$$ S^2 = \frac{\sum(X-\overline X)^2}{N-1} $$

But I don't see how can we consider the Bessel's correction of a sample of a continuous random variable, specially if the probability measure is not the same for each observation.

$$ \operatorname{Var}(X) = \sigma^2 = \int (x-\mu)^2 f(x)\, dx $$

I searched in google and in this website to no avail.

Thanks for your help, as always.

$\endgroup$
6
  • 2
    $\begingroup$ Bessels correction only makes sense for sample variances. It is used to ensure that when you use a sample to estimate the population variance, that the estimation is unbiased. If you know the population, you can just compute it’s variance, you don’t need to estimate it. $\endgroup$ Jun 20 '19 at 18:31
  • $\begingroup$ The sample variance of a continuous random variable, I meant. How can I reflect the Bessel's correction? $\endgroup$ Jun 20 '19 at 19:14
  • 1
    $\begingroup$ I don't quite follow. If you have a continuous random variable, say a normally distributed one, then have a sample from it, and want to unbiasedly estimate the population variance using the sample, then you may use Bessel's correction. $\endgroup$ Jun 20 '19 at 20:55
  • 4
    $\begingroup$ I find your question confusing. 1. Samples are discrete, not continuous (if you think otherwise, can you give an example?) and each observation is $\frac{1}{n}$-th of the sample. 2. Bessel's correction applies when you replace $\mu$ by $\bar{X}$. $\endgroup$
    – Glen_b
    Jun 20 '19 at 22:25
  • 1
    $\begingroup$ Bessel's correction consists of dividing by $N-1$ rather than by $N$ when one writes $$ \widehat\sigma^2 = \frac{\sum_{i=1}^N (\,X_i - \overline X\,)^2}{N-1}. $$ That is true regardess of whether the probability distribution of $X_i$ is discrete or continuous. If one somehow knew the value of $\mu$, one could write $$ \widehat \sigma^2 = \frac{\sum_{i=1}^N (\,X_i - \mu\,)^2}{N} $$ and it would be an unbiased estimator of $\sigma^2$ with $N$ rather than $N-1$ in the denominator. So Bessel's correction is intended to correct the bias introduced by the use of $\overline X$ rather than $\mu.\,\,$ $\endgroup$ Jun 21 '19 at 2:47
2
$\begingroup$

Sample variance: The variance $S^2$ of a random sample $X_1, X_2, \dots, X_n$ from a population with variance $\sigma^2$ is usually defined as $$S^2 = \frac{1}{n-1}\sum_{i=1}^n (X_i - \bar X)^2,$$ where $\bar X =\frac 1n \sum_{i=1}^n X_i.$ The use of $n-1$ instead of $n$ in the denominator of $S^2$ makes $S^2$ an unbiased estimator or $\sigma^2;$ that is, $E(S^2) = \sigma^2.$

Furthermore, if the data are from a normal distribution we have $$\frac {(n-1)S^2} {\sigma^2} \sim \mathsf{Chisq}(\nu = n-1),$$

a relationship used to make confidence intervals for $\sigma^2$ and to do tests involving $\sigma^2$ based on $S^2.$

Sample standard deviation: The sample standard deviation is usually defined as $$S = \sqrt{S^2} = \sqrt{\frac{1}{n-1}\sum_{i=1}^n (X_i - \bar X)^2}.$$

Because expectation is a linear operator and taking the square root is not a lineat transformation, we do not generally have $E(S) = \sigma,$ so $S$ is not an unbiased estimate of $\sigma.$

For a normal sample of size $n,$ the exact relationship is $$E(S_n) = \sigma\sqrt{\frac{2}{n-1}}\Gamma\left(\frac n2\right)/ \Gamma\left(\frac{n-1}{2}\right),$$ where $\Gamma(\cdot)$ is the gamma function. Thus for a random sample of size $n = 5$ from a normal population with standard deviation $\sigma,$ we have $E(S_5) \approx 0.940 \sigma.$ For $n = 100,\,$ $E(S_{100}) \approx 0.9975\sigma.$ Computations in R:

sqrt(2/4)*gamma(5/2)/gamma(4/2)
[1] 0.9399856
sqrt(2/99)*gamma(100/2)/gamma(99/2)
[1] 0.997478

For small $n,$ the bias is not large enough to be a difficulty in many applications, and for large $n,$ the bias is often ignored.

Addendum on the estimation of $\sigma^2:$ It seems that there are compromises to be made all around in making inferences about normal population variances.

A popular criterion for judging the usefulness of an estimator is 'root mean square error' (RMSE). The RMSE of an estimator $T$ of a parameter $\tau$ is defined as $\sqrt{E[(T-\tau))^2]}.$ A small RMSE is considered desirable.

With $Q = \sum_i (X_i - \bar X)^2,$ denote the sample variance $V_1 = S^2 = Q/(n-1),$ the MLE as $V_2 = Q/n.$ Also, $V_3 = Q/(n+1)$ and $V_4 = Q/(n+2).$

According the RMSE criterion, the sample variance $V_1 = S^2$ has a slightly larger RMSE than the MLE $V_1,$ so one might argue in favor of using the MLE. However, $V_3$ has still smaller RMSE, but its use is resisted because it is even more biased than the MLE.

For the case $n = 10, \sigma = 15, \sigma^2 = 225,$ the following simulation illustrates some of the properties of these estimators. (Estimator $V_4$ in included just to show that $Q/(n+2)$ has a larger RMSE than does $V_3.)$

set.seed(620);  n = 10;  sg = 15;  m = 10^6
v1 = replicate(m, var(rnorm(n,0,sg)))
v2 = ((n-1)/n)*v1;  v3 = ((n-1)/(n+1))*v1
v4 = ((n-1)/(n+2))*v1

mean(v1); mean(v2); mean(v3); mean(v4)
[1] 225.0488   # aprx E(S) = 225
[1] 202.5439
[1] 184.1308
[1] 168.7866

sqrt(mean((v1-sg^2)^2))
[1] 106.05              # RMSE of MLE                     
sqrt(mean((v2-sg^2)^2))
[1] 98.05116            # RMSE of S
sqrt(mean((v3-sg^2)^2))
[1] 95.91148            # smallest of 4 RMSEs
sqrt(mean((v4-sg^2)^2))
[1] 97.39696

Histograms of the simulated distributions of the four variance estimators, with vertical bars at $\sigma^2 = 225.$

enter image description here

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.