24
$\begingroup$

So far, I've been using the Shapiro-Wilk statistic in order to test normality assumptions in small samples.

Could you please recommend another technique?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Here are a couple of other questions of possible interest: is-normality-testing-essentially-useless, for a discussion of the value of normality testing, & what-if-residuals-are-normally-distributed-but-y-is-not, for a discussion / clarification of the sense in which normality is an assumption of a linear model. $\endgroup$
    – gung - Reinstate Monica
    Commented Sep 13, 2012 at 19:53
  • 3
    $\begingroup$ The Wilk in Shapiro-Wilk refers to Martin B. Wilk. It's all too easy to write "Wilks" especially (a) if someone else said or wrote that and you're copying (b) you know about the work in statistics of Samuel S. Wilks, a quite different person (c) you get confused about terminal "s" in English, given its other uses for plurals (statistics, cats, dogs, ...) and possessives ('s), which is common even among those whose first language is English. I've edited this thread to the extent I can; I can't reach into comments. $\endgroup$
    – Nick Cox
    Commented Feb 22, 2015 at 12:35

7 Answers 7

Reset to default
26
$\begingroup$

The fBasics package in R (part of Rmetrics) includes several normality tests, covering many of the popular frequentist tests -- Kolmogorov-Smirnov, Shapiro-Wilk, Jarque–Bera, and D'Agostino -- along with a wrapper for the normality tests in the nortest package -- Anderson–Darling, Cramer–von Mises, Lilliefors (Kolmogorov-Smirnov), Pearson chi–square, and Shapiro–Francia. The package documentation also provides all the important references. Here is a demo that shows how to use the tests from nortest.

One approach, if you have the time, is to use more than one test and check for agreement. The tests vary in a number of ways, so it isn't entirely straightforward to choose "the best". What do other researchers in your field use? This can vary and it may be best to stick with the accepted methods so that others will accept your work. I frequently use the Jarque-Bera test, partly for that reason, and Anderson–Darling for comparison.

You can look at "Comparison of Tests for Univariate Normality" (Seier 2002) and "A comparison of various tests of normality" (Yazici; Yolacan 2007) for a comparison and discussion of the issues.

It's also trivial to test these methods for comparison in R, thanks to all the distribution functions. Here's a simple example with simulated data (I won't print out the results to save space), although a more full exposition would be required:

library(fBasics); library(ggplot2)
set.seed(1)

# normal distribution
x1 <- rnorm(1e+06)   
x1.samp <- sample(x1, 200)
qplot(x1.samp, geom="histogram")
jbTest(x1.samp)
adTest(x1.samp)

# cauchy distribution
x2 <- rcauchy(1e+06)
x2.samp <- sample(x2, 200)
qplot(x2.samp, geom="histogram")
jbTest(x2.samp)
adTest(x2.samp)

Once you have the results from the various tests over different distributions, you can compare which were the most effective. For instance, the p-value for the Jarque-Bera test above returned 0.276 for the normal distribution (accepting) and < 2.2e-16 for the cauchy (rejecting the null hypothesis).

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks Shane, great answer! Well, "the others" from my field often use SPSS, so they use Kolmogorov-Smirnov (if they check normality at all), though IMHO the Lilliefors' test is a better choice when the data is gathered from a sample (when parameters are unknown). I was taught that Shapiro-Wilk's is appropriate for small samples, and just wanted to get more info about "small samples normality tests"... BTW, I use nortest in R! =) $\endgroup$
    – aL3xa
    Commented Aug 13, 2010 at 16:11
17
$\begingroup$

For normality, actual Shapiro-Wilk has good power in fairly small samples.

The main competitor in studies that I have seen is the more general Anderson-Darling, which does fairly well, but I wouldn't say it was better. If you can clarify what alternatives interest you, possibly a better statistic would be more obvious. [edit: if you estimate parameters, the A-D test should be adjusted for that.]

[I strongly recommend against considering Jarque-Bera in small samples (which probably better known as Bowman-Shenton in statistical circles - they studied the small sample distribution). The asymptotic joint distribution of skewness and kurtosis is nothing like the small-sample distribution - in the same way a banana doesn't look much like an orange. It also has very low power against some interesting alternatives - for example it has low power to pick up a symmetric bimodal distribution that has kurtosis close to that of a normal distribution.]

Frequently people test goodness of fit for what turn out to be not-particularly-good reasons, or they're answering a question other than the one that they actually want to answer.

For example, you almost certainly already know your data aren't really normal (not exactly), so there's no point in trying to answer a question you know the answer to - and the hypothesis test doesn't actually answer it anyway.

Given you know you don't have exact normality already, your hypothesis test of normality is really giving you an answer to a question closer to "is my sample size large enough to pick up the amount of non-normality that I have", while the real question you're interested in answering is usually closer to "what is the impact of this non-normality on these other things I'm interested in?". The hypothesis test is measuring sample size, while the question you're interested in answering is not very dependent on sample size.

There are times when testing of normality makes some sense, but those situations almost never occur with small samples.

Why are you testing normality?

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks for a great answer, and a great question afterwards. It's crucial to get an insight about the background of the problem. Well, so many times I've seen people doing t-test, Pearson's r or ANOVA without getting any idea about the shape of distribution (which is often heavy-skewed) - parametric techniques "need" satisfied normality assumption. In psychology (which is my field of interest), we often deal with small samples, therefore I need appropriate normality test. $\endgroup$
    – aL3xa
    Commented Aug 17, 2010 at 16:41
  • 6
    $\begingroup$ But normality is never satisfied. It's sometimes a reasonable description of the data, but they're not actually normal.While it's sensible to check for non-normality when you assume it, it's not particularly useful to test it (for the reasons I described above). I do a qq-plot, for example, but a hypothesis test answers the wrong question in this situation. t-tests and anova usually work reasonably well if the distributions aren't heavily skew. A better approach might be to use procedures that don't assume normality - perhaps resampling techniques. $\endgroup$
    – Glen_b
    Commented Aug 18, 2010 at 0:50
  • $\begingroup$ Or you can use non-parametric tests, at cost of having less power. And nothing is absolutely satisfied in statistics, it's not solely a normality issue. However, bootstrapping or jackknifing are not a solution when introducing someone to t-test and/or ANOVA assumptions. I doubt that resampling techniques solve normality issues at all. One should check normality both graphically (density plot, boxplot, QQplot, histogram) and "numerically" (normality tests, skewness, kurtosis, etc.). What do you suggest? This is completely off topic, but how would you check, say, ANOVA normality assumptions? $\endgroup$
    – aL3xa
    Commented Aug 18, 2010 at 1:55
  • $\begingroup$ @aL3xa I think the randomization approach is better appropriate given your research field; notwithstanding the fact that usual parametric tests provide good approximation to exact permutation tests, non-parametric tests also imply some kind of assumption (e.g. on shape of the distribution). I even wonder how we might really define what is a deviation from normality in small-sample study. I think you should ask for further discussion about this point in a separate question. $\endgroup$
    – chl
    Commented Sep 16, 2010 at 17:35
15
$\begingroup$

There is a whole Wikipedia category on normality tests including:

I think A-D is probably the best of them.

Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ I agree. I performed a quick test of the A-D test, Jarque-Bera, and Spiegelhalter's test (1983), under the null, with sample size 8, repeating 10,000 times. The A-D test maintains nominal rejection rate, and gives uniform pvals, while J-B test is terrible, Spiegelhalter is middling. $\endgroup$
    – shabbychef
    Commented Aug 13, 2010 at 17:18
  • 1
    $\begingroup$ @shabbychef The Jarque-Bera test relies on asymptotic normality of sample skewness and kurtosis, which doesn't work well even for n in the low 100s. But to obtain the desired rejection rate you can adjust critical values eg based on simulation results, as in Section 4.1 of Thadewald, T, and H. Buning, 2004, Jarque-Bera test and its competitors for testing normality - A power comparison, Discussion Paper Economics 2004/9, School of Business and Economics, Free University of Berlin. $\endgroup$
    – Silverfish
    Commented Dec 26, 2014 at 20:18
6
$\begingroup$

For completeness, econometricians also like the Kiefer and Salmon test from their 1983 paper in Economics Letters -- it sums 'normalized' expressions of skewness and kurtosis which is then chi-square distributed. I have an old C++ version I wrote during grad school I could translate into R.

Edit: And here is recent paper by Bierens (re-)deriving Jarque-Bera and Kiefer-Salmon.

Edit 2: I looked over the old code, and it seems that it really is the same test between Jarque-Bera and Kiefer-Salmon.

Improve this answer
$\endgroup$
3
$\begingroup$

In fact the Kiefer Salmon test and the Jarque Bera test are critically different as shown in several places but most recently here -Moment Tests for Standardized Error Distributions: A Simple Robust Approach by Yi-Ting Chen. The Kiefer Salmon test by construction is robust in the face of ARCH type error structures unlike the standard Jarque Bera test. The paper by Yi-Ting Chen develops and discusses what I think are likely to be the best tests around at the moment.

Improve this answer
$\endgroup$
1
  • 4
    $\begingroup$ Chen seems to focus on larger datasets, which makes sense because the fourth and sixth and higher moments involved in these tests are going to take some time to settle down to asymptotic levels. But distributional tests are typically used for datasets smaller than 250 values (the minimum studied in this paper). In fact, most of them become so powerful with larger amounts of data that they are little more than afterthoughts in such applications. Or is there more going on here than I am seeing? $\endgroup$
    – whuber ♦
    Commented Nov 6, 2010 at 15:18
0
$\begingroup$

For sample sizes <30 subjects, Shapiro-Wilk is considered to have a robust power - Be careful, when adjusting the significance level of the test, since it may induce a type II error! [1]

Improve this answer
$\endgroup$
2
  • $\begingroup$ In small samples goodness of fit tests are generally unable to reject normaility. $\endgroup$
    – Michael R. Chernick
    Commented Apr 12, 2017 at 17:42
  • $\begingroup$ @MichaelChernick what happens in the specific case then? What is the reason behind a small small being "classified" as non-normal? $\endgroup$
    – Aliakbar Ahmadi
    Commented Apr 12, 2017 at 21:36
0
$\begingroup$

I read about the z-score intervals:

For small samples (n < 50), if absolute z-scores for either skewness or kurtosis are larger than 1.96, which corresponds with an alpha level 0.05, then reject the null hypothesis and conclude the distribution of the sample is non-normal

(Hae-Young Kim, 2013.)

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.