0
$\begingroup$

I'm new to statistics and reading stats books I found different definitions for Variance.

Definition1: $ s^2 = \frac{1}{n-1} \sum_{i=1}^n (x_i - \bar{x})^2 $

Definition2: $ s^2 = \frac{1}{n} \sum_{i=1}^n (x_i - \bar{x})^2 $

I would like to know why is used different definition for same thing?

$\endgroup$
  • $\begingroup$ They are similar ideas, but the 1st is the sample variance and the 2nd is a special case of the population variance where $x$ takes on discrete values with equal probability (or perhaps it's a biased estimate of the population variance. I would need context). $\endgroup$ – Matthew Gunn Oct 21 '16 at 18:59
  • $\begingroup$ Check also this recent thread stats.stackexchange.com/questions/239379/… $\endgroup$ – Tim Oct 21 '16 at 19:01
  • 2
    $\begingroup$ This question is essentially the same as stats.stackexchange.com/questions/3931: although the emphasis is different, it is thoroughly answered within many of the replies there. $\endgroup$ – whuber Oct 21 '16 at 19:34
  • $\begingroup$ What one may add to all of this is that when the population is finite then dividing by $n$ gives the population variance when all elements of the population are observed. All this discussion below is presented for the case when we are dealing with theoretically infinite populations. $\endgroup$ – tomka Oct 22 '16 at 9:19
3
$\begingroup$

Your first definition is a sample variance. The second might also be the sample variance or it might be a calculation of the variance for a discrete valued random variable with equiprobable outcomes. (saying more would require context.)

Population Variance:

The population variance (or simply variance) is defined as:

$$ \mathrm{Var}(x) = \mathrm{E}\left[ \left( x - \mathrm{E}[x] \right)^2 \right] $$

Special case ($x$ is a discrete valued random variable):

For a random variable $x$ which takes only discrete values, the expectation is a sum and can be written as:

$$ \mathrm{Var}(x) = \sum_i p_i (x_i - \mathrm{E}[x])^2 \quad \quad \mathrm{E}[x] = \sum_i p_i x_i$$ where $p_i$ is the probability of outcome $x_i$.

Even more special case:

A further special case is when each outcome is equally probable (eg. a dice role) in which case $p_i = \frac{1}{n}$ hence $ \mathrm{Var}(x) = \frac{1}{n} \sum_i (x_i - \mathrm{E}[x])^2 $ where $\mathrm{E}[x] = \frac{1}{n}\sum_i x_i$

Sample Variance:

Let's say you have $n$ independent and identically distributed observations $x_i$. Your set of $\{x_i\}$ form a sample. The sample mean is given by:

$$ \bar{x} = \frac{1}{n} \sum_{i=1}^n x_i$$

The sample mean is the mean of the sample. It is also an estimate of the population mean $E[x]$.

Similarly, we can compute the sample variance. $$\frac{1}{n-1} \sum_i \left(x_i - \bar{x} \right)^2 \quad \text{ where } \quad \bar{x} = \frac{1}{n} \sum x_i$$

The $n-1$ here is a degrees of freedom correction. If you're confused by it, don't worry too much. It's a technical correction so that expectation of the sample variance is the population variance. The idea is that your estimate of the sample variance is based upon $n-1$ observations instead of $n$ (intuition: if you had $n=1$ observations, then $x_1 = \bar{x}$! In some sense, one degree of freedom in your datagets used up when you demean your series (i.e. subtract off the sample mean).

Another technical note is that sometimes $\frac{1}{n} \sum_i \left( x_i - \bar{x} \right)^2$ is used as the sample variance (i.e. not everyone makes the degrees of freedom correction and in large samples, it doesn't matter).

Wikipedia has a whole section on population vs. sample variance here.

| cite | improve this answer | |
$\endgroup$
1
$\begingroup$

The variance is a kind of average squared difference from the mean.

When you have the entire population of values, that's in fact exactly its definition.

However with samples, you don't know the population mean, so you must use some estimate -- using the sample mean in the variance formula actually minimizes the average squared difference from the mean, making the sum of squares smaller than if you used the true population mean. So when you use the sample mean to calculate an estimate of the variance, it's too small. As it turns out, the resulting variance estimate is on average $\frac{n-1}{n}$ what it should be, so if you multiply by $\frac{n}{n-1}$ you make it the right size on average.

So a common thing to do with sample variances is to cancel out the two terms in $n$ (the denominator of the average squared distance with the numerator of the adjustment) so that you end up dividing by $n-1$ instead of $n$ when you calculate the average.

This use of $n-1$ on the denominator instead of $n$ is known as Bessel's correction. This is named after Bessel because it was first used by Gauss in 1823 (see Stigler's Law).

[Both definitions are perfectly valid ways to define a sample variance (we use biased estimators all the time). Indeed, if you're interested in estimating standard deviation (which is often the case) both the $n$ and $n-1$ divisor will yield biased estimates.]

| cite | improve this answer | |
$\endgroup$
0
$\begingroup$

It's important to notice that what you've written there is the variance of n equally likely discrete random values (you want to get familiar with what these things). Generally speaking the Variance is the Expected Value of the squared deviation from the mean of X. $$ s^2=E[(X- \bar x)^2] $$ Where the expected value, E, is the "long-term mean" of the repetitions of experiments (e.g. the expected value of getting head flipping a coin is 0.5). So if every x, generated from X, is equally likely, then you get its mean from: $$ \bar x={1\over n}\sum_{i=1}^n x_i $$ And the variance is: $$ s^2={1\over n}\sum_{i=1}^n (x_i-\bar x)^2 $$ I don't really know why you'd want to divide everything by n-1, giving what I've said earlier.

| cite | improve this answer | |
$\endgroup$
  • 2
    $\begingroup$ The $\frac{n}{n-1}$ correction is a technical, degrees of freedom correction so that the expectation of the sample variance is equal to the actual variance. $\endgroup$ – Matthew Gunn Oct 21 '16 at 19:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.