# What is the difference between the Shapiro-Wilk test of normality and the Kolmogorov-Smirnov test of normality?

What is the difference between the Shapiro-Wilk test of normality and the Kolmogorov-Smirnov test of normality? When will results from these two methods differ?

-

You can't really even compare the two since the Kolmogorov-Smirnov is for a completely specified distribution (so if you're testing normality, you must specify the mean and variance; they can't be estimated from the data*), while the Shapiro-Wilk is for normality, with unspecified mean and variance.

* you also can't scale by parameter estimates and test for standard normal.

One way to compare would be to supplement the Shapiro-Wilk with a test for specified mean and variance in a normal (combining the tests in some manner), or by having the KS tables adjusted for the parameter estimation (but then it's no longer distribution-free).

There is such a test (equivalent to the Kolmogorov-Smirnov with estimated parameters) - the Lilliefors test; the normality-test version could be validly compared to the Shapiro-Wilk (and will generally have lower power). More competitive is the Anderson-Darling test (which must also be adjusted for parameter estimation for a comparison to be valid).

As for what they test - the KS test (and the Lilliefors) looks at the largest difference between the empirical CDF and the specified distribution, while the Shapiro Wilk effectively compares two estimates of variance; the closely related Shapiro-Francia can be regarded as a monotonic function of the squared correlation in a Q-Q plot; if I recall correctly, the Shapiro-Wilk also takes into account covariances between the order statistics.

Edited to add: While the Shapiro-Wilk nearly always beats the Lilliefors test on alternatives of interest, an example where it doesn't is the $t_{30}$ in medium-large samples ($n>60$-ish). There the Lilliefors has higher power.

-
This is an interesting answer, but I am having a little trouble understanding how to square it with practice. Maybe these should be different questions, but what is the consequence of ignoring the parameter estimation in the K-S test? Does this imply that the Lillefors test has less power than an incorrectly conducted K-S in which the pareters were estimated from the data? –  rpierce Nov 13 '13 at 20:42
@rpierce - The main impact of treating estimated parameters as known is to dramatically lower the actual significance level (and hence the power curve) from what it should be if you take account of it (as the Lilliefors does). That is, the Lilliefors is the K-S 'done right' for parameter estimation and it has substantially better power than the KS. On the other hand, the Lilliefors has much worse power than say the Shapiro-Wilk test. In short, the KS isn't an especially powerful test to start with, and we make it worse by ignoring that we're doing parameter estimation. –  Glen_b Nov 13 '13 at 22:11
... keeping in mind when we say 'better power' and 'worse power' that we're generally referring to power against what people generally regard as interesting sorts of alternatives. –  Glen_b Nov 13 '13 at 22:18
Just to clarify, what you mean is that the SW is more sensitive to departures from normality than the KS (even though the KS shouldn't be used in this way) and the KS is more sensitive than the Lillefors to departures from normality? –  rpierce Nov 27 '13 at 13:46
How do you get the second conclusion? Now I'm worried I have misspoken somewhere. –  Glen_b Nov 27 '13 at 17:33

Briefly stated, the Shapiro-Wilk test is a specific test for normality, whereas the method used by Kolmogorov-Smirnov test is more general, but less powerful (meaning it correctly rejects the null hypothesis of normality less often). Both statistics take normality as the null and establishes a test statistic based on the sample, but how they do so is different from one another in ways that make them more or less sensitive to features of normal distributions.

How exactly W (the test statistic for Shapiro-Wilk) is calculated is a bit involved, but conceptually, it involves arraying the sample values by size and measuring fit against expected means, variances and covariances. These multiple comparisons against normality, as I understand it, give the test more power than the the Kolmogorov-Smirnov test, which is one way in which they may differ.

By contrast, the Kolmogorov-Smirnov test for normality is derived from a general approach for assessing goodness of fit by comparing the expected cumulative distribution against the empirical cumulative distribution, vis:

As such, it is sensitive at the center of the distribution, and not the tails. However, the K-S is test is convergent, in the sense that as n tends to infinity, the test converges to the true answer in probability (I believe that Glivenko-Cantelli Theorem applies here, but someone may correct me). These are two more ways in which these two tests might differ in their evaluation of normality.

-
Besides... Shapiro-Wilk's test is often used when estimating departures from normality in small samples. Great answer, John! Thanks. –  aL3xa Jul 30 '10 at 1:24
+1, two other notes about KS: it can be used to test against any major distribution (whereas SW is only for normality), & the lower power could be a good thing w/ larger samples. –  gung Jul 26 '12 at 16:49
How is lower power a good thing? As long as Type I remains the same isn't higher power always better? Furthermore, KS is not generally less powerful, only maybe to leptokurtosis? For example, KS is much more powerful for skew without a commensurate increase in Type 1 errors. –  John Nov 24 '12 at 9:14
The Kolmogorov-Smirnov is for a fully specified distribution. The Shapiro Wilk is not. They can't be compared ... because as soon as you make the adjustments required to make them comparable, you no longer have one or the other test. –  Glen_b Nov 8 '13 at 8:18
Found this simulation study, in case that adds anything useful in the way of details. Same general conclusion as above: the Shapiro-Wilk test is more sensitive. ukm.my/jsm/pdf_files/SM-PDF-40-6-2011/15%20NorAishah.pdf –  Nick Stauner Nov 8 '13 at 16:01