8
$\begingroup$

Out of curiosity, I realised that the Kolmogorov-Smirnov normality test returns two very different p-values depending on whether the dataset has small or large numbers. Is this normal and is there a number size limit for this test? From what I saw, the Shapiro-Wilk test was much more stable.

I tried this

ks.test(c(0.5379796,1.1230795,-0.4047321,-0.8150001,0.9706860),"pnorm")

One-sample Kolmogorov-Smirnov test

data:  c(0.5379796, 1.1230795, -0.4047321, -0.8150001, 0.970686)

D = 0.3047, p-value = 0.6454

alternative hypothesis: two-sided

And then I multiplied each value by 100



ks.test(c(53.79796,112.30795,-40.47321,-81.50001,97.06860),"pnorm")

    One-sample Kolmogorov-Smirnov test

data:  c(53.79796, 112.30795, -40.47321, -81.50001, 97.06860)

D = 0.6, p-value = 0.03008

alternative hypothesis: two-sided

With the same data, the Shapiro-Wilk test returns a p-value of 0.3999.

$\endgroup$
2
  • $\begingroup$ Okay, those numbers look more like you multiplied to 100. :) $\endgroup$
    – Dave
    Commented Mar 6, 2023 at 9:43
  • $\begingroup$ This question is addressed in multiple posts already on site. $\endgroup$
    – Glen_b
    Commented Mar 7, 2023 at 23:10

2 Answers 2

17
$\begingroup$

The ways you’ve coded it, you’re asking the KS test about a null hypothesis that the distribution is $N(0,1)$. In the first set of numbers, that looks plausible. Consequently, the p-value is high. In the second set of numbers, that does not seem to be the case. Numbers like those don’t typically come from a $N(0,1)$ distribution. Consequently, the p-value is low.

By multiplying by a factor, you’ve changed the variance. Since the KS test considers all aspects of the distribution, variance included, the test correctly regards the two data sets as different.

The reason that Shapiro-Wilk is more stable is because it evaluates normality. Multiplying by a positive factor does not change the normality, so Shapiro-Wilk will not have the same kind of sensitivity to a variance change that KS has.

$\endgroup$
8
  • 1
    $\begingroup$ True, there was a typo in the code for the *100. but your answer still holds, thank you ! $\endgroup$
    – Xav64
    Commented Mar 6, 2023 at 9:44
  • 3
    $\begingroup$ In addition, there are specific warnings about using the K-S test with parameters estimated from the data, which you presumably need here, and general warnings about the relative uselessness of any such tests. Just plot the data on a normal quantile plot! (Many other names: normal probability plot, normal scores plot, etc.) $\endgroup$
    – Nick Cox
    Commented Mar 6, 2023 at 9:54
  • 2
    $\begingroup$ See this for elaboration on what @NickCox wrote about normality testing being of questionable usefulness. // Regarding issues with testing with estimated parameters, check out the Lilliefors test. $\endgroup$
    – Dave
    Commented Mar 6, 2023 at 9:57
  • 1
    $\begingroup$ @Xav64 Standardizing your data with the sample mean and variance and then testing if the standardized data are $N(0,1)$ is the same as testing with the original data and using the sample mean and variance. You can check this in a simulation where you will get identical p-values. (I assume you mean the usual z-score by standardizing: $x_{new}=\dfrac{x_{original}-\bar x}{s}$.) $\endgroup$
    – Dave
    Commented Mar 6, 2023 at 9:58
  • 2
    $\begingroup$ I don’t totally follow what you’re asking, but that Lilliefors test might be what you want. @Xav64 $\endgroup$
    – Dave
    Commented Mar 6, 2023 at 10:09
8
$\begingroup$

Adding to the existing response, it's worth noting that the two ks.test calls below produce the same output.

x = c(0.5379796,1.1230795,-0.4047321,-0.8150001,0.9706860)
ks.test(x, pnorm)
#> 
#>  Exact one-sample Kolmogorov-Smirnov test
#> 
#> data:  x
#> D = 0.3047, p-value = 0.6454
#> alternative hypothesis: two-sided

ks.test(x*100, pnorm, sd = 100)
#> 
#>  Exact one-sample Kolmogorov-Smirnov test
#> 
#> data:  x * 100
#> D = 0.3047, p-value = 0.6454
#> alternative hypothesis: two-sided

R syntax note: the default arguments to pnorm() are mean = 0, sd = 1. Anything after the second argument in ks.test() gets passed as an argument to the pnorm() function in this case.

$\endgroup$
2
  • $\begingroup$ so does it mean that the optimal way of running this test is to write ks.test(x, pnorm, mean=mean(sample), sd=sd(sample)) or is there a faster way? $\endgroup$
    – Xav64
    Commented Mar 6, 2023 at 12:04
  • 6
    $\begingroup$ Reply to comment: 1. No because that test is invalid. See remarks elsewhere with key word=:Lilliefors. 2. Who said optimal=faster? $\endgroup$
    – Nick Cox
    Commented Mar 6, 2023 at 12:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.