10
$\begingroup$

I am trying to understand when to use Shapiro-Wilk Test and/or Kolmogorov-Smirnov Test for normality. Currently, I am following this website How to Test for Normality in R (4 Methods). This paper Descriptive Statistics and Normality Tests for Statistical Data says:

The Shapiro–Wilk test is more appropriate method for small sample sizes (<50 samples) although it can also be handling on larger sample size while Kolmogorov–Smirnov test is used for n ≥50. For both of the above tests, null hypothesis states that data are taken from normal distributed population.

So it suggests using the Kolmogorov-Smirnov test for a sample size bigger than 50 is more appropriate. While the website I mentioned above, uses the same sample size for both tests to check normality. I don't understand why

Also from this What is the difference between the Shapiro–Wilk test of normality and the Kolmogorov–Smirnov test of normality? and answer, it seems that you can't actually compare the tests.

So I was wondering why the website is comparing the same sample size for normality and when is it not possible to use Shapiro-Wilk test anymore?

$\endgroup$
6
  • 3
    $\begingroup$ It is worth flagging that the first link details two graphical methods before it gets to those testing methods. I endorse that emphasi I would strongly endorse the use of normal quantile plots. I wouldn't trust myself, or even any statistics expert, to be able to use histograms to tell normality from say a Student's t distribution with about 7 df (which is what the late great Sir Harold Jeffreys found was more typical even for measurement errors in datasets of high quality). $\endgroup$
    – Nick Cox
    Commented Dec 12, 2022 at 12:48
  • 7
    $\begingroup$ The second link gets confused between variance and SD. That is not a good sign. $\endgroup$
    – Nick Cox
    Commented Dec 12, 2022 at 13:03
  • 3
    $\begingroup$ "when to use Shapiro-Wilk Test and/or Kolmogorov-Smirnov Test for normality?" Use them when answering questions from inadequately-trained instructors or paper reviewers. Do NOT use them when determining how to approach regression problems. The threat to validity is generally from issues regarding homoschedasticity with the predictor set or non-linear relationships between predictors and outcomes. $\endgroup$
    – DWin
    Commented Dec 13, 2022 at 1:28
  • $\begingroup$ @Nick If you mean the paper from which the quotation is taken, I agree. "Central limit theorem states that when sample size has 100 or more observations, violation of the normality is not a major issue" says it all. It reads like a computer-age update of G. Gamow's book 1,2,3, Infinity, wherein he muses on a culture that has no names for numbers greater than 3. (Compare Terry Pratchett's trolls, who use "one, two, lots, many" -- but, unlike humans, manage to extend that using base 4 numbers.) Evidently, for these authors, $100$ is indistinguishable from infinity. $\endgroup$
    – whuber
    Commented Dec 13, 2022 at 21:15
  • $\begingroup$ @whuber I was alluding to the square root in the displayed variance formula. I did not read the entire paper carefully. $\endgroup$
    – Nick Cox
    Commented Dec 13, 2022 at 21:19

2 Answers 2

12
$\begingroup$

The main difference between the S-W and the K-S test for normality is that the S-W can be used to assess the goodness of fit to a fitted distribution, whereas the K-S test is only valid to test against a prespecified distribution.

So if you estimate the parameters of your distribution from your data, then test the goodness of fit of your data against the distribution with the estimated mean and variance, you can't use the K-S any more - it will have a too optimistic view of the goodness of fit. In this situation, use the S-W test. (Normalizing your data with a mean and variance estimated from the data is the same thing in this context.) If, conversely, you know the precise distribution your data should be coming from (e.g., an N(5,1) distribution), you can use K-S.

Thus, the rationale for using one test over another does not hang on the sample size, at least as long as the distinction above is not kept in mind. Once this is kept in mind, you can start looking at power against specific alternatives.

There is a lot of statistical misinformation floating around the internet.

(Incidentally, there is much less necessity for testing normality than is commonly assumed, see the replies to this chat message:.

How to cope with non-normal ANOVA residuals?

Assumptions of linear models and what to do if the residuals are not normally distributed

Analysis of variance with not normally distributed residuals : how important is normality? )


Edit: John Madden asks a very interesting question:

Wouldn't large sample sizes allow for hand-waving away the fact that parameters had to be estimated by standard arguments (i.e. continuous mapping theorem?), and perhaps this is where the idea that KS is more appropriate for large samples comes from?

Let's take a look. We will simulate $x_1, \dots, x_n\sim N(0,1)$, for increasing sample sizes $n$. For this vector $x$, we perform a Kolmogorov-Smirnov test against a normal distribution with mean and standard deviation equal to the mean and standard deviation of $x$, and store the $p$ value. For each $n$, we do this simulation exercise 10,000 times and plot the $p$ values in a histogram. If the K-S test were valid, this histogram would be uniform, and if John's question had a positive answer, the histograms would become more uniform as $n$ increases. So we run the exercise for $n\in\{10,100,1000,10000\}$. The histograms don't get any more uniform, so no, the K-S-test against estimated parameters does not get better for large sample sizes:

histograms

R code:

exponents <- 1:4
par(mfrow=c(2,2),mai=c(.5,.1,.5,.1))
for ( ee in exponents ) {
    p_values <- rep(NA,1e4)
    for ( ii in seq_along(p_values) ) {
        sims <- rnorm(10^ee)
        p_values[ii] <- ks.test(sims,pnorm,mean=mean(sims),sd=sd(sims))$p.value
    }
    hist(p_values,xlab="",yaxt="n",
        main=paste("Histogram of p values\nSample size:",10^ee))
}

(It turns out that whuber already did this analysis almost a year ago.)

As various commenters point out completely correctly, there are different ways of addressing this property of the K-S test, like the Lilliefors modification, or bootstrapping the test statistic obtained with estimated parameters.

$\endgroup$
8
  • 4
    $\begingroup$ Your comments are correct about the original K-S test. But Lilliefor's extension of that test lets it be used when the mean and SD are estimated from the sample (not assumed about the population): Dallal GE and Wilkinson L (1986), "An Analytic Approximation to the Distribution of Lilliefors's Test Statistic for Normality," The American Statistician, 40, 294-296. $\endgroup$ Commented Dec 12, 2022 at 16:04
  • 1
    $\begingroup$ @HarveyMotulsky: quite correct. But that variation is often called the "Lilliefors test" and treated separately from the K-S and the S-W test (as it should be IMO, to avoid potential confusion). $\endgroup$ Commented Dec 12, 2022 at 16:14
  • 3
    $\begingroup$ @StephanKolassa: Yet when people bootstrap the distribution of the K-S statistic from a best-fit null they call that a "Kolmogorov-Smirnov test". Whatever position you take on the naming of tests, it's always a good idea to mention Lilliefors tests if you're saying the K-S test needs a prespecified distribution under the null. $\endgroup$ Commented Dec 12, 2022 at 20:13
  • 3
    $\begingroup$ @JohnMadden: that is a very interesting question, and the answer seems to be "no", take a look at my edit. $\endgroup$ Commented Dec 13, 2022 at 8:21
  • 2
    $\begingroup$ @John I posted the results of the same study a year ago at stats.stackexchange.com/a/564922/919 ;-). $\endgroup$
    – whuber
    Commented Dec 13, 2022 at 21:07
8
$\begingroup$

I presume that by the Kolmogorov-Smirnov they actually mean the Lilliefors test -- the estimated-parameter version for the normal case that was tabulated by Lilliefors; he did tables for the exponential as well. This (calling Lilliefors' modification for estimated parameters "Kolmogorov-Smirnov") seems to come mostly from SPSS; many authors follow its lead. This has been the cause of many confusions.

The Lilliefors can be compared to the Shapiro Wilk, since they're both tests of normality with unspecified parameters.

Which one is preferable depends on which alternatives you seek power against (and, occasionally, sample size can make some difference).

Often in the sorts of situations people tend to consider in power comparisons - but not always - the Shapiro-Wilk will come out with higher power. In some such cases the Lilliefors has better power, though.

The Shapiro–Wilk test is more appropriate method for small sample sizes (<50 samples) although it can also be handling on larger sample size while Kolmogorov–Smirnov test is used for n ≥50. For both of the above tests, null hypothesis states that data are taken from normal distributed population.

This is poor advice; you can find it in many places, but it stems from the simple fact that Shapiro and Wilk themselves only produced tables to $n=50$, over half a century ago, as if nobody had done any work on it since.

Naturally, that is not the case. Computer functions that will work for much, much larger $n$ have been available for decades. The version of the test that is implemented in R goes out to $n=5000$, using Patrick Royston's algorithms from papers published in 1982 and 1995. The usual advice when you go beyond the end of available Shapiro-Wilk functionality is to use a Shapiro-Francia test.

There's nothing that happens at $n=50$ for a wide variety of alternatives that would change the considerations of which test you might use. If you were in some situation where the Shapiro-Wilk is the better test at $n=45$, it's (nearly always) going to be the better option at $n=55$ and $n=75$ as well.

It's important to be aware that there are dozens of tests that could be used besides these two, and if broad power against likely alternatives of interest is what you care about, I would tend to avoid both these tests as there are other tests that do typically better in most of the situations these do well at (among the situations that people tend to worry about). If you like the Shapiro-Wilk, why not Chen-Shapiro? If you like ECDF-related tests, why not the version of the Anderson-Darling adjusted for parameter estimation? There are many other possibilities besides.

I will add my support for Nick Cox's comments above. Graphical methods come closer to addressing relevant considerations if you're worried about whether the normality assumption for some test should be of concern (at least if you're in a situation where avoiding letting the data choose the hypothesis is impossible to avoid, which would be fairly rare).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.