How can we ever know the population variance?

Question

In hypothesis testing, a common question is what is the population variance? My question is how can we ever know the population variance? If we knew the entire distribution, we might as well know the mean of the entire population. Then what is the point of hypothesis testing?

Answer 1

I'm not sure that this issue really comes up "often" outside of Stats 101 (introduction to statistics). I'm not sure I've ever seen it. On the other hand, we do present the material that way when teaching introductory courses, because it provides a logical progression: You start with a simple situation where there is only one group and you know the variance, then progress to where you don't know the variance, then progress to where there are two groups (but with equal variance), etc.

To address a slightly different point, you ask why we would bother with hypothesis testing if we knew the variance, since we must therefore also know the mean. The latter part is reasonable, but the first part is a misunderstanding: The mean we would know would be the mean under the null hypothesis. That's what we're testing. Consider @StephanKolassa's example of IQ scores. We know the mean is 100 and the standard deviation is 15; what we're testing is if our group (say, left-handed redheads, or perhaps introductory statistics students) differs from that.

Answer 2

Often we don't know the population variance as such - but we have a very reliable estimate from a different sample. For instance, here is an example on assessing whether average weight of penguins has gone down, where we use the mean from a small-ish sample, but the variance from a larger independent sample. Of course, this presupposes that the variance is the same in both populations.

A different example might be classical IQ scales. These are normalized to have a mean of 100 and a standard deviation of 15, using really large samples. We might then take a specific sample (say, 50 left-handed redheads) and ask whether their mean IQ is significantly larger than 100, using 15^2 as a "known" variance. Of course, once again, this begs the question whether the variance is really equal between the two samples - after all, we are already testing whether means are different, so why should variances be equal?

Bottom line: your concerns are valid, and usually tests with known moments only serve didactic purposes. In statistics courses, they are usually immediately followed with tests using estimated moments.

Answer 3

The only way to know the population variance is to measure the entire population.

However, measuring an entire population is often not feasible; it requires resources including money, tools, personnel, and access. For this reason we sample populations; that is measuring a subset of the population. The sampling process should be designed carefully and with the objective of creating a sample population which is representative of the population; giving two key considerations - sample size and sampling technique.

Toy example: You wish to estimate the variance in weight for the adult population of Sweden. There are some 9.5 million Swedes so it is not likely that you can go out and measure them all. Therefore you need to measure a sample population from which you can estimate the true within-population variance.

You head out to sample the Swedish population. To do this you go and stand in Stockholm city centre, and just so happen to stand right outside the popular fictitious Swedish burger chain Burger Kungen. In fact, it's raining and cold (it must be summer) so you stand inside the restaurant. Here you weigh four people.

The chances are, your sample will not reflect the population of Sweden very well. What you have is a sample of people in Stockholm, who are in a burger restaurant. This is a poor sampling technique because it is likely to bias the result by not giving a fair representation of the population which you are trying to estimate. Furthermore, you have a small sample size, so you have a high risk of picking four people that are in the extremes of the population; either very light or very heavy. If you sampled 1000 people you are less likely to cause a sampling bias; it is far less likely to pick 1000 people that are unusual than it is to pick four that are unusual. A larger sample size would at least give you a more accurate estimate of the mean and variance in weight among the customers of Burger Kungen.

The histogram illustrates the effect of sampling technique, the grey distribution could represent the population of Sweden that doesn't eat at Burger Kungen (mean 85 kg), while the red could represent the population of the customers of Burger Kungen (mean 100 kg), and the blue dashes could be the four people you sample. Correct sampling technique would need to weigh the population fairly, and in this case ~75% of the population, thus 75% of the samples that are measured, should not be customers of Burger Kungen.

This is a major issue with a lot of surveys. For example, people likely to respond to surveys of customer satisfaction, or opinion polls in elections, tend to be disproportionately represented by those with extreme views; people with less strong opinions tend to be more reserved in expressing them.

The point of hypothesis testing is (not always), for example, to test whether two populations differ from one another. E.g. Do customers of Burger Kungen weigh more than Swedes that do not eat at Burger Kungen? The ability to test this accurately is reliant on proper sampling technique and sufficient sample size.

---

R code to test make all this happen:

df1 = data.frame(rnorm(9500000, 85, 15), sample(c("Y","N","N","N"), replace = T))
colnames(df1) = c("weight","customer")
df1$weight = ifelse(df1$customer == "Y", df1$weight + rnorm(length(df1$weight[df1$customer =="Y"]), 15, 2), df1$weight)
subsample = sample(df1$weight[df1$customer=="Y"], size = 4)

png(paste0(path,"SwedenWeight.png"), res =1000, width = 4, height = 4, units = "in")
par(mar=c(5,6,2,2))
hist(df1$weight[df1$customer=="N"], xlab = "Kilograms", col = rgb(0,0,0,0.5), main ="")
hist(df1$weight[df1$customer=="Y"], add = T, col = rgb(1,0,0,0.5))
axis(side = 1, at = c(subsample), labels = c("","","",""), tck = -0.03, col = "blue")
axis(side = 1, at = c(0,150), labels = c("",""), tck = -0)
dev.off()

t.test(df1$weight~df1$customer)

Results:

> t.test(df1$weight~df1$customer)

        Welch Two Sample t-test

data:  df1$weight by df1$customer
t = -1327.7, df = 4042400, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -15.04688 -15.00252
sample estimates:
mean in group N mean in group Y 
       84.99555       100.02024 

Answer 4

Sometimes the population variance is set a priori. For example, SAT scores are scaled so that the standard deviation is 110 and IQ tests are scaled to have a standard deviation of 15.

Answer 5

The only realistic example I can think of when the mean is unknown but the variance is known is when there is random sampling of points on a hypersphere (in whatever dimension) with a fixed radius and an unknown centre. This problem has an unknown mean (centre of the sphere) but a fixed variance (squared-radius of the sphere). I am unaware of any other realistic examples where there is an unknown mean but known variance. (And to be clear: merely having an outside variance estimate from other data is not an example of a known variance. Also, if you have this variance estimate from other data, why don't you also have a corresponding mean estimate from that same data?)

In my view, introductory statistical courses that teach tests with an unknown mean and known variance are an anachronism, and they are misguided as a modern teaching tool. Pedagogically, it is far better to start directly with the T-test for the case of an unknown mean and variance, and treat the z-test as an asymptotic approximation to this that holds when the degrees-of-freedom is large (or not even bother to teach the z-test at all). The number of situations where there would be a known variance but unknown mean is vanishingly small, and it is generally misleading to students to introduce this (insanely rare) case.

Answer 6

Sometimes in applied problems, there are reasons presented by physics, economics, etc that tell us about variance and have no uncertainty. Other times, the population may be finite and we may happen to know some things about everyone, but need to sample and perform statistics to learn the rest.

Generally, your concern is pretty valid.