Why are Shapiro-Wilk and Kolmogorov-Smirnov test compared with same sample size?

Question

I am trying to understand when to use Shapiro-Wilk Test and/or Kolmogorov-Smirnov Test for normality. Currently, I am following this website How to Test for Normality in R (4 Methods). This paper Descriptive Statistics and Normality Tests for Statistical Data says:

The Shapiro–Wilk test is more appropriate method for small sample sizes (<50 samples) although it can also be handling on larger sample size while Kolmogorov–Smirnov test is used for n ≥50. For both of the above tests, null hypothesis states that data are taken from normal distributed population.

So it suggests using the Kolmogorov-Smirnov test for a sample size bigger than 50 is more appropriate. While the website I mentioned above, uses the same sample size for both tests to check normality. I don't understand why

Also from this What is the difference between the Shapiro–Wilk test of normality and the Kolmogorov–Smirnov test of normality? and answer, it seems that you can't actually compare the tests.

So I was wondering why the website is comparing the same sample size for normality and when is it not possible to use Shapiro-Wilk test anymore?

Answer 1

The main difference between the S-W and the K-S test for normality is that the S-W can be used to assess the goodness of fit to a fitted distribution, whereas the K-S test is only valid to test against a prespecified distribution.

So if you estimate the parameters of your distribution from your data, then test the goodness of fit of your data against the distribution with the estimated mean and variance, you can't use the K-S any more - it will have a too optimistic view of the goodness of fit. In this situation, use the S-W test. (Normalizing your data with a mean and variance estimated from the data is the same thing in this context.) If, conversely, you know the precise distribution your data should be coming from (e.g., an N(5,1) distribution), you can use K-S.

Thus, the rationale for using one test over another does not hang on the sample size, at least as long as the distinction above is not kept in mind. Once this is kept in mind, you can start looking at power against specific alternatives.

There is a lot of statistical misinformation floating around the internet.

(Incidentally, there is much less necessity for testing normality than is commonly assumed, see the replies to this chat message:.

How to cope with non-normal ANOVA residuals?

Assumptions of linear models and what to do if the residuals are not normally distributed

Analysis of variance with not normally distributed residuals : how important is normality? )

---

Edit: John Madden asks a very interesting question:

Wouldn't large sample sizes allow for hand-waving away the fact that parameters had to be estimated by standard arguments (i.e. continuous mapping theorem?), and perhaps this is where the idea that KS is more appropriate for large samples comes from?

Let's take a look. We will simulate $x_1, \dots, x_n\sim N(0,1)$, for increasing sample sizes $n$. For this vector $x$, we perform a Kolmogorov-Smirnov test against a normal distribution with mean and standard deviation equal to the mean and standard deviation of $x$, and store the $p$ value. For each $n$, we do this simulation exercise 10,000 times and plot the $p$ values in a histogram. If the K-S test were valid, this histogram would be uniform, and if John's question had a positive answer, the histograms would become more uniform as $n$ increases. So we run the exercise for $n\in\{10,100,1000,10000\}$. The histograms don't get any more uniform, so no, the K-S-test against estimated parameters does not get better for large sample sizes:

R code:

exponents <- 1:4
par(mfrow=c(2,2),mai=c(.5,.1,.5,.1))
for ( ee in exponents ) {
    p_values <- rep(NA,1e4)
    for ( ii in seq_along(p_values) ) {
        sims <- rnorm(10^ee)
        p_values[ii] <- ks.test(sims,pnorm,mean=mean(sims),sd=sd(sims))$p.value
    }
    hist(p_values,xlab="",yaxt="n",
        main=paste("Histogram of p values\nSample size:",10^ee))
}

(It turns out that whuber already did this analysis almost a year ago.)

As various commenters point out completely correctly, there are different ways of addressing this property of the K-S test, like the Lilliefors modification, or bootstrapping the test statistic obtained with estimated parameters.

Answer 2

I presume that by the Kolmogorov-Smirnov they actually mean the Lilliefors test -- the estimated-parameter version for the normal case that was tabulated by Lilliefors; he did tables for the exponential as well. This (calling Lilliefors' modification for estimated parameters "Kolmogorov-Smirnov") seems to come mostly from SPSS; many authors follow its lead. This has been the cause of many confusions.

The Lilliefors can be compared to the Shapiro Wilk, since they're both tests of normality with unspecified parameters.

Which one is preferable depends on which alternatives you seek power against (and, occasionally, sample size can make some difference).

Often in the sorts of situations people tend to consider in power comparisons - but not always - the Shapiro-Wilk will come out with higher power. In some such cases the Lilliefors has better power, though.

The Shapiro–Wilk test is more appropriate method for small sample sizes (<50 samples) although it can also be handling on larger sample size while Kolmogorov–Smirnov test is used for n ≥50. For both of the above tests, null hypothesis states that data are taken from normal distributed population.

This is poor advice; you can find it in many places, but it stems from the simple fact that Shapiro and Wilk themselves only produced tables to $n=50$, over half a century ago, as if nobody had done any work on it since.

Naturally, that is not the case. Computer functions that will work for much, much larger $n$ have been available for decades. The version of the test that is implemented in R goes out to $n=5000$, using Patrick Royston's algorithms from papers published in 1982 and 1995. The usual advice when you go beyond the end of available Shapiro-Wilk functionality is to use a Shapiro-Francia test.

There's nothing that happens at $n=50$ for a wide variety of alternatives that would change the considerations of which test you might use. If you were in some situation where the Shapiro-Wilk is the better test at $n=45$, it's (nearly always) going to be the better option at $n=55$ and $n=75$ as well.

It's important to be aware that there are dozens of tests that could be used besides these two, and if broad power against likely alternatives of interest is what you care about, I would tend to avoid both these tests as there are other tests that do typically better in most of the situations these do well at (among the situations that people tend to worry about). If you like the Shapiro-Wilk, why not Chen-Shapiro? If you like ECDF-related tests, why not the version of the Anderson-Darling adjusted for parameter estimation? There are many other possibilities besides.

I will add my support for Nick Cox's comments above. Graphical methods come closer to addressing relevant considerations if you're worried about whether the normality assumption for some test should be of concern (at least if you're in a situation where avoiding letting the data choose the hypothesis is impossible to avoid, which would be fairly rare).