Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I'm new to statistics and reading stats books I found different definitions for Variance.
Definition1: $ s^2 = \frac{1}{n-1} \sum_{i=1}^n (x_i - \bar{x})^2 $
Definition2: $ s^2 = \frac{1}{n} \sum_{i=1}^n (x_i - \bar{x})^2 $
I would like to know why is used different definition for same thing?
Your first definition is a sample variance. The second might also be the sample variance or it might be a calculation of the variance for a discrete valued random variable with equiprobable outcomes. (saying more would require context.)
The population variance (or simply variance) is defined as:
$$ \mathrm{Var}(x) = \mathrm{E}\left[ \left( x - \mathrm{E}[x] \right)^2 \right] $$
For a random variable $x$ which takes only discrete values, the expectation is a sum and can be written as:
$$ \mathrm{Var}(x) = \sum_i p_i (x_i - \mathrm{E}[x])^2 \quad \quad \mathrm{E}[x] = \sum_i p_i x_i$$ where $p_i$ is the probability of outcome $x_i$.
A further special case is when each outcome is equally probable (eg. a dice role) in which case $p_i = \frac{1}{n}$ hence $ \mathrm{Var}(x) = \frac{1}{n} \sum_i (x_i - \mathrm{E}[x])^2 $ where $\mathrm{E}[x] = \frac{1}{n}\sum_i x_i$
Let's say you have $n$ independent and identically distributed observations $x_i$. Your set of $\{x_i\}$ form a sample. The sample mean is given by:
$$ \bar{x} = \frac{1}{n} \sum_{i=1}^n x_i$$
The sample mean is the mean of the sample. It is also an estimate of the population mean $E[x]$.
Similarly, we can compute the sample variance. $$\frac{1}{n-1} \sum_i \left(x_i - \bar{x} \right)^2 \quad \text{ where } \quad \bar{x} = \frac{1}{n} \sum x_i$$
The $n-1$ here is a degrees of freedom correction. If you're confused by it, don't worry too much. It's a technical correction so that expectation of the sample variance is the population variance. The idea is that your estimate of the sample variance is based upon $n-1$ observations instead of $n$ (intuition: if you had $n=1$ observations, then $x_1 = \bar{x}$! In some sense, one degree of freedom in your datagets used up when you demean your series (i.e. subtract off the sample mean).
Another technical note is that sometimes $\frac{1}{n} \sum_i \left( x_i - \bar{x} \right)^2$ is used as the sample variance (i.e. not everyone makes the degrees of freedom correction and in large samples, it doesn't matter).
Wikipedia has a whole section on population vs. sample variance here.
The variance is a kind of average squared difference from the mean.
When you have the entire population of values, that's in fact exactly its definition.
However with samples, you don't know the population mean, so you must use some estimate -- using the sample mean in the variance formula actually minimizes the average squared difference from the mean, making the sum of squares smaller than if you used the true population mean. So when you use the sample mean to calculate an estimate of the variance, it's too small. As it turns out, the resulting variance estimate is on average $\frac{n-1}{n}$ what it should be, so if you multiply by $\frac{n}{n-1}$ you make it the right size on average.
So a common thing to do with sample variances is to cancel out the two terms in $n$ (the denominator of the average squared distance with the numerator of the adjustment) so that you end up dividing by $n-1$ instead of $n$ when you calculate the average.
This use of $n-1$ on the denominator instead of $n$ is known as Bessel's correction. This is named after Bessel because it was first used by Gauss in 1823 (see Stigler's Law).
[Both definitions are perfectly valid ways to define a sample variance (we use biased estimators all the time). Indeed, if you're interested in estimating standard deviation (which is often the case) both the $n$ and $n-1$ divisor will yield biased estimates.]
It's important to notice that what you've written there is the variance of n equally likely discrete random values (you want to get familiar with what these things). Generally speaking the Variance is the Expected Value of the squared deviation from the mean of X. $$ s^2=E[(X- \bar x)^2] $$ Where the expected value, E, is the "long-term mean" of the repetitions of experiments (e.g. the expected value of getting head flipping a coin is 0.5). So if every x, generated from X, is equally likely, then you get its mean from: $$ \bar x={1\over n}\sum_{i=1}^n x_i $$ And the variance is: $$ s^2={1\over n}\sum_{i=1}^n (x_i-\bar x)^2 $$ I don't really know why you'd want to divide everything by n-1, giving what I've said earlier.
