2
$\begingroup$

Example from Wikipedia:

Suppose that in a particular geographic region, the mean and standard deviation of scores on a reading test are 100 points, and 12 points, respectively. Our interest is in the scores of 55 students in a particular school who received a mean score of 96. We can ask whether this mean score is significantly lower than the regional mean—that is, are the students in this school comparable to a simple random sample of 55 students from the region as a whole, or are their scores surprisingly low?

They further use population sigma as the standard deviation:

$\mathrm{SE} = \frac{\sigma}{\sqrt n} = \frac{12}{\sqrt{55}} = \frac{12}{7.42} = 1.62 \,\!$

This particular region may have higher variance in scores (for example, half of students are from poor families and have to work instead of studying). Why here (and in other text books) they assume variance as equal to population variance?

Another example: Example #2. We are OK when means of population and sample are different, but we do not let variances to be different. Also here you can see it directly. Is not it a misleading?

Yes, I know that we should use one sample t-test, and I am OK with calculation of variance using true known mean under $H_0$, but the question is about Z-test which is usually taught before t-test.

UPD Probable Answer: It seems that if you have population with $SD = x$ and you subsample from population with $SD != x$ - then your population is not normally distributed, but you have a mix of normal distributions (even if the means are the same). Correct me if I am wrong. Another problem that we do not care if population is normally distributed - we care that subsample is normally distributed (if we want to make Z-test stronger). But it is another story and have nothing in common with the assumptions of original Z test...

UPD: Wrong again. Assume we subsample guys who did not finish the school and have low scores in reading test.

scores <- rnorm(10000, mean=100, sd = 12)
region_scores <- scores[which(scores < 90)]

Distribution of region scores is not normal any more. We can not apply Z-test.

It seems that we can never apply the Z-test. Or, if we satisfy the test's assumptions and sample from the population randomly, we will have nothing interesting to test - our subsample will deviate in mean randomly, and no true effect will arise.

Test assumptions are described here. Why do they want to estimate deviation in means with random sampling from a normal population? There will be no true difference in means! I'm totally confused.

$\endgroup$
  • 1
    $\begingroup$ Good anticipation of a possible problem! You can let your estimates of the variance be different for the poor and the rich. This would allow you to estimate what's known as Heteroscedasticity-consistent standard errors. $\endgroup$ – Matthew Gunn May 6 '16 at 10:58
  • $\begingroup$ Thanks for the answer. Yes, I will explore data at first, I will test for variance and (hopefully) my test will be powerful enough to detect the difference. But the question is: why they teach it in this way? Is it right to say that all the textbooks are basically wrong? What is the motivation of this assumption - simplicity's sake? Can I say "I do not recommend to use one-sample Z-test in practice even for large sample sizes"? $\endgroup$ – German Demidov May 6 '16 at 11:12
  • $\begingroup$ And another question: how to test for difference in variances when I have a population of 1 million and sample of 40 samples? Standard tests with 999999 degrees of freedom look kinda stupid. $\endgroup$ – German Demidov May 6 '16 at 11:14
  • $\begingroup$ I think this is a very interesting question - my gut feeling is that the answer is rooted in historical developments, as well as the natural progression in the academic syllabus. Also in the ease of defaulting to simpler models. Clearly in the majority of the real-life situations the t test is more realistic, because we don't know $\sigma$. Even more honest is to assume that the populations $\sigma$'s can differ. $\endgroup$ – Antoni Parellada May 6 '16 at 12:52
  • $\begingroup$ It would be interesting to get a canonical answer, although I think that discussing the Behrens-Fisher problem would be pertinent. $\endgroup$ – Antoni Parellada May 6 '16 at 12:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.