Is there a rule of thumb to choose a normality test?

Question

Is there a rule which normality test a junior statistician should use in different situations. I read from Wikipedia that there are so many. And what one can do if there is a limit case such that one test shows that it is reasonable to assume the normality hypothesis and another test says one should reject the same hypothesis? Or can this kind of situation ever appear?

Answer 1

There are many dozens of tests of goodness of fit* which are either usable as a form of test of normality or could be adapted to it.

*(I believe there's something over a hundred that can be found in the literature, though counts would depend on exactly how you divide them up)

In many situations, the most appropriate test of normality (Is normality testing 'essentially useless'?) is not to do a hypothesis test at all.

Goodness of fit hypothesis tests don't assess whether it is "reasonable to assume normality", and rejection doesn't necessarily imply that a normality assumption is unreasonable for some purpose.

Indeed as sample sizes become large you become almost certain to reject normality, but that may hardly affect many typical forms of inference that assume normality. On the other hand, at small sample size, you'll rarely reject, but the impact of the non-normality you can't detect may be substantial, so non-rejection is little use to you. Even at intermediate sample sizes, rejection or non-rejection of some particular test may not be very helpful in understanding whether it's of much concern for a particular application.

Which is to say, they really don't address a question you're generally interested in -- those questions are generally better addressed by looking at how much things* might be impacted by the amount of (and kind of) non-normality that's present.

* i.e. whatever so far unstated things you might be trying to achieve

Different tests of normality do have sensitivity to different kinds of departures from normality (for example an Anderson-Darling-like test will be more sensitive to deviations in the tail, compared to say a Kolmogorov-Smirnov/Lilliefors type test), the problem is that they don't tell you about effect size (how big the deviation is), nor its impact (how much it affects whatever you wanted to check normality for).

Various tests also have different 'blind spots' where one or another test is very insensitive to particular kinds of departure from non-normality. Indeed, test bias - an even lower rejection rate than $\alpha$ when the null is false in some way - is very common with goodness of fit tests.

There are a variety of approaches to making such assessments, of which simulation is often relatively simple to use. These assessments allow investigation of what kinds of departures are most important and how big their effect might be.

The Shapiro Wilk test is a good omnibus test that's available in many packages, and if you actually need a generic test of normality it's a reasonably powerful choice against a variety of interesting alternatives ... but the issues discussed above all apply.