10
$\begingroup$

In hypothesis testing, a common question is what is the population variance? My question is how can we ever know the population variance? If we knew the entire distribution, we might as well know the mean of the entire population. Then what is the point of hypothesis testing?

$\endgroup$
  • $\begingroup$ Some relevant literature: nber.org/papers/w20325 $\endgroup$ – dv_bn May 25 '16 at 11:47
  • $\begingroup$ One can know the variance without knowing anything about the mean. For instance, the variance can be recovered from the squares of all differences of values in the population, but those differences give no information about the mean. Regardless, I do not see how the statements and questions in this post lead up to the question itself about the point of hypothesis testing. $\endgroup$ – whuber May 25 '16 at 20:14
10
$\begingroup$

I'm not sure that this issue really comes up "often" outside of Stats 101 (introduction to statistics). I'm not sure I've ever seen it. On the other hand, we do present the material that way when teaching introductory courses, because it provides a logical progression: You start with a simple situation where there is only one group and you know the variance, then progress to where you don't know the variance, then progress to where there are two groups (but with equal variance), etc.

To address a slightly different point, you ask why we would bother with hypothesis testing if we knew the variance, since we must therefore also know the mean. The latter part is reasonable, but the first part is a misunderstanding: The mean we would know would be the mean under the null hypothesis. That's what we're testing. Consider @StephanKolassa's example of IQ scores. We know the mean is 100 and the standard deviation is 15; what we're testing is if our group (say, left-handed redheads, or perhaps introductory statistics students) differs from that.

$\endgroup$
  • 2
    $\begingroup$ (+1) Perhaps it comes up more when "sampling from a population" is a way of thinking about the data-generating process, rather than something to be taken literally. Knowing the precision of a measuring instrument for example. $\endgroup$ – Scortchi - Reinstate Monica May 25 '16 at 13:37
  • $\begingroup$ Gung, as a practitioner with a 20+ year career, this issue came up in my experience more frequently than you imply. I'm not suggesting that it came up "frequently," just that the debates occurred. However, and to your point about Stats 101, more times than not the discussions were red herrings that resolved little or nothing regarding the particulars of a study or project -- somebody just wanted to create the appearance of intelligence in asking the question. $\endgroup$ – Mike Hunter May 25 '16 at 15:34
  • 1
    $\begingroup$ @DJohnson, I suppose it depends on the topics you work on. $\endgroup$ – gung - Reinstate Monica May 25 '16 at 15:44
4
$\begingroup$

Often we don't know the population variance as such - but we have a very reliable estimate from a different sample. For instance, here is an example on assessing whether average weight of penguins has gone down, where we use the mean from a small-ish sample, but the variance from a larger independent sample. Of course, this presupposes that the variance is the same in both populations.

A different example might be classical IQ scales. These are normalized to have a mean of 100 and a standard deviation of 15, using really large samples. We might then take a specific sample (say, 50 left-handed redheads) and ask whether their mean IQ is significantly larger than 100, using 15^2 as a "known" variance. Of course, once again, this begs the question whether the variance is really equal between the two samples - after all, we are already testing whether means are different, so why should variances be equal?

Bottom line: your concerns are valid, and usually tests with known moments only serve didactic purposes. In statistics courses, they are usually immediately followed with tests using estimated moments.

$\endgroup$
2
$\begingroup$

The only way to know the population variance is to measure the entire population.

However, measuring an entire population is often not feasible; it requires resources including money, tools, personnel, and access. For this reason we sample populations; that is measuring a subset of the population. The sampling process should be designed carefully and with the objective of creating a sample population which is representative of the population; giving two key considerations - sample size and sampling technique.

Toy example: You wish to estimate the variance in weight for the adult population of Sweden. There are some 9.5 million Swedes so it is not likely that you can go out and measure them all. Therefore you need to measure a sample population from which you can estimate the true within-population variance.

You head out to sample the Swedish population. To do this you go and stand in Stockholm city centre, and just so happen to stand right outside the popular fictitious Swedish burger chain Burger Kungen. In fact, it's raining and cold (it must be summer) so you stand inside the restaurant. Here you weigh four people.

The chances are, your sample will not reflect the population of Sweden very well. What you have is a sample of people in Stockholm, who are in a burger restaurant. This is a poor sampling technique because it is likely to bias the result by not giving a fair representation of the population which you are trying to estimate. Furthermore, you have a small sample size, so you have a high risk of picking four people that are in the extremes of the population; either very light or very heavy. If you sampled 1000 people you are less likely to cause a sampling bias; it is far less likely to pick 1000 people that are unusual than it is to pick four that are unusual. A larger sample size would at least give you a more accurate estimate of the mean and variance in weight among the customers of Burger Kungen.

enter image description here

The histogram illustrates the effect of sampling technique, the grey distribution could represent the population of Sweden that doesn't eat at Burger Kungen (mean 85 kg), while the red could represent the population of the customers of Burger Kungen (mean 100 kg), and the blue dashes could be the four people you sample. Correct sampling technique would need to weigh the population fairly, and in this case ~75% of the population, thus 75% of the samples that are measured, should not be customers of Burger Kungen.

This is a major issue with a lot of surveys. For example, people likely to respond to surveys of customer satisfaction, or opinion polls in elections, tend to be disproportionately represented by those with extreme views; people with less strong opinions tend to be more reserved in expressing them.

The point of hypothesis testing is (not always), for example, to test whether two populations differ from one another. E.g. Do customers of Burger Kungen weigh more than Swedes that do not eat at Burger Kungen? The ability to test this accurately is reliant on proper sampling technique and sufficient sample size.


R code to test make all this happen:

df1 = data.frame(rnorm(9500000, 85, 15), sample(c("Y","N","N","N"), replace = T))
colnames(df1) = c("weight","customer")
df1$weight = ifelse(df1$customer == "Y", df1$weight + rnorm(length(df1$weight[df1$customer =="Y"]), 15, 2), df1$weight)
subsample = sample(df1$weight[df1$customer=="Y"], size = 4)

png(paste0(path,"SwedenWeight.png"), res =1000, width = 4, height = 4, units = "in")
par(mar=c(5,6,2,2))
hist(df1$weight[df1$customer=="N"], xlab = "Kilograms", col = rgb(0,0,0,0.5), main ="")
hist(df1$weight[df1$customer=="Y"], add = T, col = rgb(1,0,0,0.5))
axis(side = 1, at = c(subsample), labels = c("","","",""), tck = -0.03, col = "blue")
axis(side = 1, at = c(0,150), labels = c("",""), tck = -0)
dev.off()

t.test(df1$weight~df1$customer)

Results:

> t.test(df1$weight~df1$customer)

        Welch Two Sample t-test

data:  df1$weight by df1$customer
t = -1327.7, df = 4042400, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -15.04688 -15.00252
sample estimates:
mean in group N mean in group Y 
       84.99555       100.02024 
$\endgroup$
1
$\begingroup$

Sometimes the population variance is set a priori. For example, SAT scores are scaled so that the standard deviation is 110 and IQ tests are scaled to have a standard deviation of 15.

$\endgroup$
  • $\begingroup$ Yes, that's true, but in those cases there is also scaling to a fixed mean, so it does not yield a situation where there is an unknown mean and known variance. Also, the scaling is done after all values are known. $\endgroup$ – Reinstate Monica Feb 19 at 9:25
1
$\begingroup$

The only realistic example I can think of when the mean is unknown but the variance is known is when there is random sampling of points on a hypersphere (in whatever dimension) with a fixed radius and an unknown centre. This problem has an unknown mean (centre of the sphere) but a fixed variance (squared-radius of the sphere). I am unaware of any other realistic examples where there is an unknown mean but known variance. (And to be clear: merely having an outside variance estimate from other data is not an example of a known variance. Also, if you have this variance estimate from other data, why don't you also have a corresponding mean estimate from that same data?)

In my view, introductory statistical courses that teach tests with an unknown mean and known variance are an anachronism, and they are misguided as a modern teaching tool. Pedagogically, it is far better to start directly with the T-test for the case of an unknown mean and variance, and treat the z-test as an asymptotic approximation to this that holds when the degrees-of-freedom is large (or not even bother to teach the z-test at all). The number of situations where there would be a known variance but unknown mean is vanishingly small, and it is generally misleading to students to introduce this (insanely rare) case.

$\endgroup$
0
$\begingroup$

Sometimes in applied problems, there are reasons presented by physics, economics, etc that tell us about variance and have no uncertainty. Other times, the population may be finite and we may happen to know some things about everyone, but need to sample and perform statistics to learn the rest.

Generally, your concern is pretty valid.

$\endgroup$
  • 5
    $\begingroup$ I have a hard time picturing an example from physics or economics where we would know the variance, but not the mean. Similar for discrete distributions. Could you give a concrete example or two? $\endgroup$ – S. Kolassa - Reinstate Monica May 25 '16 at 11:40
  • $\begingroup$ @StephanKolassa I believe that physics experimental measurements would be an example - we may have a process or device of measurement that has a well known variance (measurement error), so when measuring a particular event then you can assume that variance is the same but you can only estimate the true mean. $\endgroup$ – Peteris May 25 '16 at 14:27
  • 2
    $\begingroup$ @Peteris: that makes sense - but it sounds more like the case I note, of the variance (of your instrument) having been estimated on previous "calibration samples". I'd expect a theoretically derived variance with no uncertainty (!) to be a different thing. $\endgroup$ – S. Kolassa - Reinstate Monica May 25 '16 at 14:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.