Discrete Random Variable vs Population Variance - Why Is Only The Latter Normalized?

The formula for the variance of a discrete random variable $$X$$ with values $$\{x_1,x_2, ...,x_n\}$$ and probabilities $$\{p_1,p_2,...,p_n\}$$ is given as $$\sum_{i=1}^{n}(x_i - \mu)^2p_i$$ where $$\mu = \sum_{i=1}^{n}x_ip_i$$. On the other hand the formula for variance of a population with values $$\{x_1,x_2, ...,x_n\}$$ is given as $$\frac{1}{n}\sum_{i=1}^{n}(x_i - \mu)^2$$ where $$\mu = \frac{1}{n}\sum_{i=1}^{n}x_i$$. Why is the former variance not normalized while the latter is? I assume it has to do something with the inclusion of the $$p_i$$ term in the RV's mean, but I can't tease out why.

• When I was a youngster, discreet materials arrived by mail in plain brown wrappers. ;-) Maybe that's how discreet random variables are communicated, too?
– whuber
Jun 10 '20 at 13:14

You need to recognize the distinction between $$x_i$$'s in two formulas:

Random variable. First, $$Var(X) = \sigma_X^2 = \sum_{i=1}^{k}(x_i - \mu)^2p_i,$$ where $$x_i$$'s are $$k$$ discrete values that the random variable $$X$$ may take. No two of these $$x_i$$'s may be alike.

Population. Second, $$Var(X) = \sigma_X^2 = \frac{1}{N}\sum_{i=1}^N (X_i - \mu)^2,$$ where $$X_i$$'s are the $$N$$ values appearing in the population. Some of these $$X_i$$'s may have the same value.

Urn example. Let's look at a specific example: Suppose an urn contains a population of $$N = 20$$ numbered chips. One chip has the number 1; two chips have the number 2, three have the number 3 and four have 4. That accounts for ten of the twenty chips. Also, one chip as 8, two have 7, three have 6, and four have 5. That accounts for the rest.

Then the mean of the population is $$E(X)=\mu_X = \frac{1}{20}\sum_{i=i}^{20} x_i \\ = \frac{1}{20}[1 + 2 + 2 + 3 + 3+ 3 +4+4+4+4\\+5+5+5+5+6+6+6+7+7+8]\\ = \frac{1}{20}[1 + (2+2) + (3+3+3) + (4+4+4+4)\\ + (5+5+5+5) + (6+6+6) + (7+7) + 8]\\ = \frac{1}{20}[1 + 2(2) + 3(3) + 4(4) + 4(5) + 3(6) + 2(7) + 8]\\ = \frac{1}{20}[1 + 4 + 9 + 16 + 20 + 18 + 14 + 8] = 90/20 = 4.5.$$

Similarly, the variance of the population is $$Var(X) = \sigma_X^2 = \frac{1}{20}\sum_{i=1}^{20} (x_i - \mu)^2\\ = \frac{1}{20}[(1-4.5)^2 + 2(2-4.5)^2 + \cdots + (8-4.5)^2]\\ = \frac{1}{20}[12.25 + 3(6.25) + 3(2.25)+4(0.25)\\+4(0.25)+3(2.25) +2(6.25) + 12.25]\\ = \frac{1}{20}[65] = 65/20 = 3.25.$$

f = c(1,2,3,4,4,3,2,1)
x = 1:8
sum(f*(x-4.5)^2)
[1] 65
sum(f*(x-4.5)^2)/20
[1] 3.25


Then returning to the $$k = 8$$ values of the random variable $$X$$ we have probabilities $$p_i$$ taking values $$1/20, 2/20, 3/20, 4/20, 4/20, 3/20, 2/20, 1/10$$ for the respective values of $$x_i,$$ which are $$1,2,3,4,5,6,7,8,$$ respectively. Then $$E(X) = \mu_X = \sum_{1=1}^8 x_ip_i\\ = 1(1/20)+2(2/20)+3(3/2)+4(4/20)\\+5(4/20)+6(3/2)+7(2/20) + 8(1/20)\\ = \frac{1}{20}[1 + 4 +9 +16+20+18+14+8] = 90/20 = 4.5,$$ as above.

Similarly, $$Var(X) = \sigma_X^2 = \sum_{1=1}^8 x_ip_i\\ = \cdots = \frac{1}{20}[(1-4.5)^2 + 2(2-4.5)^2 + \cdots + (8-4.5)^2]\\ = \cdots = \frac{1}{20}[65] = 65/20 = 3.25,$$ as above.

Simulation. I can simulate drawing a large sample of chips from the urn with replacement, by using R. The sample procedure in R has three parameters of interest to us. The first specifies he population, the second specifies the sample size, the third rep=T indicates sampling with replacement.\$

Simulation is cheap, so I will draw a million chips with replacement, and then find the sample mean of the numbers on chips drawn and also find the variance. With a million draws the sample mean should approximate the population mean quite well; similarly, the sample variance should match the population variance.

set.seed(2020)
pop = c(1, 2,2, 3,3,3, 4,4,4,4, 5,5,5,5, 6,6,6, 7,7, 8)
x = sample(pop, 10^6, rep=T)
mean(x)
[1] 4.500654   # aprx E(X) = 4.5
var(x)
[1] 3.245843   # aprx Var(X) = 3.25

cutp = (0:8) + .5
hist(x, prob=T, br=cutp, col="skyblue2", main="Numbers on Chips Drawn")
k = 1:8;  p = c(1,2,3,4,4,3,2,1)/20
points(k, p, col="red", pch=19)


The histogram below shows the proportions of chip numbers drawn. With a million draws, these proportions should be similar to the probability distribution of $$X$$ discussed above (shown as red dots). Within the resolution of the graph, the match is essentially perfect.

These are two different $$x_i$$ in your formulas. The first one uses all possible values of the variable and $$p_i$$ denote the probabilities of the values. You could call the formula population variance as well as long as probabilities are known.

The second formula is for $$x_i$$ denoting the observations, in fact all existing observations. For instance, these could be every grade awarded to homework assignment. In this case you have the population. The grades obviously repeat. In the first formula the grades don’t repeat because the x lists all possible grades and assigns probabilities to each