What is the difference between the Shapiro–Wilk test of normality and the Kolmogorov–Smirnov test of normality? When will results from these two methods differ?
$\begingroup$
$\endgroup$
0
asked Jul 21, 2010 at 0:24
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
14
You can't really even compare the two since the Kolmogorov-Smirnov is for a completely specified distribution (so if you're testing normality, you must specify the mean and variance; they can't be estimated from the data*), while the Shapiro-Wilk is for normality, with unspecified mean and variance.
* you also can't standardize by using estimated parameters and test for standard normal; that's actually the same thing.
One way to compare would be to supplement the Shapiro-Wilk with a test for specified mean and variance in a normal (combining the tests in some manner), or by having the KS tables adjusted for the parameter estimation (but then it's no longer distribution-free).
There is such a test (equivalent to the Kolmogorov-Smirnov with estimated parameters) - the Lilliefors test; the normality-test version could be validly compared to the Shapiro-Wilk (and will generally have lower power). More competitive is the Anderson-Darling test (which must also be adjusted for parameter estimation for a comparison to be valid).
As for what they test - the KS test (and the Lilliefors) looks at the largest difference between the empirical CDF and the specified distribution, while the Shapiro Wilk effectively compares two estimates of variance; the closely related Shapiro-Francia can be regarded as a monotonic function of the squared correlation in a Q-Q plot; if I recall correctly, the Shapiro-Wilk also takes into account covariances between the order statistics.
Edited to add: While the Shapiro-Wilk nearly always beats the Lilliefors test on alternatives of interest, an example where it doesn't is the $t_{30}$ in medium-large samples ($n>60$-ish). There the Lilliefors has higher power.
[It should be kept in mind that there are many more tests for normality that are available than these.]
-
2$\begingroup$ I've seen a power curve;I just didn't think through what a lowering or raising of it would mean and instead god stuck on about your second comment starting: "keeping in mind". Somehow I got twisted around and thought you were saying that 'better' power meant having the power curve where it 'ought' to be. That perhaps we were cheating and getting unrealistic power in the KS because we were handing it parameters that it should have been penalized for estimating (because that is what I am used to as a consequence for failing to acknowledge that a parameter comes from an estimate). $\endgroup$ Nov 27, 2013 at 22:23
-
1$\begingroup$ Not sure how I missed these comments before, but yes, calculated p-values from using the KS test with estimated parameters as if they were known/specified will tend to be too high. Try it in R:
hist(replicate(1000,ks.test(scale(rnorm(x)),pnorm)$p.value))-- if the p-values were as they should be, that would look uniform! $\endgroup$– Glen_bDec 4, 2018 at 23:16 -
1$\begingroup$ You have exactly the right thought at the end there. You can standardize if you know the population mean and variance. If you estimate either or both from the sample, that substantially alters the distribution of the test statistic (try it! code is above). Hence Lilliefors' papers in the 60s on the KS tests with estimation for the case of the normal (estimating $\mu$, estimating $\sigma$, estimating both) and also for the exponential (the test is no longer distribution-free, you have to do it for each specific case). $\endgroup$– Glen_bJan 9, 2022 at 1:37
-
1$\begingroup$ Actually that code won't do what you need, since x isn't defined there. Try:
n=30;hist(replicate(10000,ks.test(scale(rnorm(n)),pnorm)$p.value),n=100)... unless you account for the effect of estimation, the test is highly conservative. (This code takes a few seconds to run) $\endgroup$– Glen_bJan 9, 2022 at 1:47 -
1$\begingroup$ Note that I addressed the issue with standardizing in my answer (that it's the same as just specifying the sample statistics as the population statistics). $\endgroup$– Glen_bJan 9, 2022 at 1:58
$\begingroup$
$\endgroup$
5
Briefly stated, the Shapiro-Wilk test is a specific test for normality, whereas the method used by Kolmogorov-Smirnov test is more general, but less powerful (meaning it correctly rejects the null hypothesis of normality less often). Both statistics take normality as the null and establishes a test statistic based on the sample, but how they do so is different from one another in ways that make them more or less sensitive to features of normal distributions.
How exactly W (the test statistic for Shapiro-Wilk) is calculated is a bit involved, but conceptually, it involves arraying the sample values by size and measuring fit against expected means, variances and covariances. These multiple comparisons against normality, as I understand it, give the test more power than the the Kolmogorov-Smirnov test, which is one way in which they may differ.
By contrast, the Kolmogorov-Smirnov test for normality is derived from a general approach for assessing goodness of fit by comparing the expected cumulative distribution against the empirical cumulative distribution, vis:
As such, it is sensitive at the center of the distribution, and not the tails. However, the K-S is test is convergent, in the sense that as n tends to infinity, the test converges to the true answer in probability (I believe that Glivenko-Cantelli Theorem applies here, but someone may correct me). These are two more ways in which these two tests might differ in their evaluation of normality.
answered Jul 21, 2010 at 15:36
-
3$\begingroup$ Besides... Shapiro-Wilk's test is often used when estimating departures from normality in small samples. Great answer, John! Thanks. $\endgroup$– aL3xaJul 30, 2010 at 1:24
-
$\begingroup$ +1, two other notes about KS: it can be used to test against any major distribution (whereas SW is only for normality), & the lower power could be a good thing w/ larger samples. $\endgroup$ Jul 26, 2012 at 16:49
-
$\begingroup$ How is lower power a good thing? As long as Type I remains the same isn't higher power always better? Furthermore, KS is not generally less powerful, only maybe to leptokurtosis? For example, KS is much more powerful for skew without a commensurate increase in Type 1 errors. $\endgroup$– JohnNov 24, 2012 at 9:14
-
$\begingroup$ The Kolmogorov-Smirnov is for a fully specified distribution. The Shapiro Wilk is not. They can't be compared ... because as soon as you make the adjustments required to make them comparable, you no longer have one or the other test. $\endgroup$– Glen_bNov 8, 2013 at 8:18
-
$\begingroup$ Found this simulation study, in case that adds anything useful in the way of details. Same general conclusion as above: the Shapiro-Wilk test is more sensitive. ukm.my/jsm/pdf_files/SM-PDF-40-6-2011/15%20NorAishah.pdf $\endgroup$ Nov 8, 2013 at 16:01