5
$\begingroup$

For example, in linear regression, we assume the errors are normally distributed, have constant variance and have an expected value of 0. These assumptions all refer to the population. But we look at our actual data points to see if these assumptions are satisfied, Why do we do this? How do we know that results hold for the population? If we only have 10 data points, why don't we say that we don't have enough data to test whether these assumptions are satisfied?

$\endgroup$
3
$\begingroup$

I disagree with holding the opinion that the data is normally distributed unless you have statistically rejected normality. This is the procedure we follow when the goal of our research is actually to REJECT H0. It is not a procedure we should follow to test the assumptions of our statistical analysis.

What do we usually do to test for normality? There are tests, but as many people do, I do not think they are usually useful. If the sample is small the power is too low and if it it large the tests detect even small deviations from normality, which are almost always there and do not actually matter much. So, the usually thing to do is to look at the QQ-plot. Since you use "visually" I assume you are familiar with it.

Since it plots estimated quantiles against theoretical quantiles, you can expect that the estimated quantiles are asymptotically the populations quantiles and that the QQ-plot is more or less stable as the sample size increases.

If your sample size is very small (10 qualifies) you either admit not being able to assume normality, or you are able to justify it in some other way (experiences with similar data, theoretical reasons,...) . In an ideal world it's at least an assumptions that should be discussed.

$\endgroup$
2
$\begingroup$

You certainly could say that you don't have enough data to test the assumptions. Generally speaking, in significance testing we hold that there is a default position which we will continue to believe unless there is sufficient evidence to the contrary. (Somewhat odd, I agree.) This 'default position' goes by the name of 'the null hypothesis'. Thus, for example with the normality assumption, we simply assume the data (actually the residuals) are normally distributed until the data force us to change our mind.

As to the question of why we do this visually instead of via a formal hypothesis test, there are several things. First, your visual system is just amazingly powerful: possibly the majority of your brain is devoted to visual processing (depending on how you count), ~70% of the sensory input is visual in nature, etc. It is considerably more powerful than the rational / reasoning parts (as counter-intuitive as that may sound). Personally, I feel like I understand something about my data when I see it that I just don't when I read in the statistical output that p<.05. I think a second reason is that there is a decent argument to be made that many statistical tests would ultimately show 'significance' if we had enough data, and thus you are only testing your $N$ (which you already knew). Moreover, if you have enough data to establish that they aren't normal, but your data are reasonably normal-ish, the central limit theorem will cover you anyway. So, what you really want to know is do you have a moderately-sized (or larger) deviation from normal with a mid-sized (or smaller) data set. Given that you know your $N$, a qq-plot, or similar approach, is more helpful. More along these lines can be found in this classic CV question.

$\endgroup$
2
  • $\begingroup$ Am I correct in understanding that if I want to show normality with a hypothesis test I start with a normal null hypothesis and only reject it if there is sufficient evidence against it? Wouldn't this be contrary to the scope of the null hypothesis? - from what I read, you should only use hypothesis testing to gather evidence in favour of the alternate hypothesis and not the null hypothesis. $\endgroup$ – Andrew Apr 25 '12 at 6:34
  • 1
    $\begingroup$ It's true that hypothesis tests are typically used when people want to show that they are justified in rejecting the null hypothesis. However, that is not always the case (another example might be nested model tests). At any rate, if you plug your data into a normal distribution test, examine the output, & if p>.05 say that your data are normal, then you are following the standard hypothesis testing procedure. Hypothesis testing is really not a good fit for this task, as I state above, but it very much does happen. (IMHO, hypothesis testing is rarely a good approach.) $\endgroup$ – gung - Reinstate Monica Apr 25 '12 at 14:16
2
$\begingroup$

Why do we look at the sample - because it's all we've got. It would be great to look at the population to see if it meets our assumption but we can't.

We typically know what the residuals (or whatever) from our sample would look like if the population met our assumptions - so we look at them, just as part of the normal inference from sample to population.

Obviously it is silly to look at your sample of 2 and conclude "yes, that's plausibly from a normal distribution". So the approach becomes one of thinking carefully about power, what can be expected from your sample size, what you know about how the sample was generated, etc.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.