2
$\begingroup$

I'm using 50,000 values of data. I can't perform a K.Wallis test in R because my data amount is bigger than 5000. Therefore I considered to use Kolmogorov-Smirnov test.

I guess that my data follows a normal distribution. However, I would like to be sure using the Ks.test. In R script I wrote:

ks.test(dat$Si,"pnorm",alternative = c("two.sided"),exact = NULL)

R results are:

data:  dat$Si 
D = 0.904, p-value < 2.2e-16
alternative hypothesis: two-sided 

Mensajes de aviso perdidos
In ks.test(dat$Si, "pnorm", alternative = c("two.sided"), exact = NULL) :
ties should not be present for the Kolmogorov-Smirnov test

What does it mean? My result follows a normal distribution or not? I guess that If I have this small p-value, it means that I must reject the Null Hypothesis. So they don't follow a normal distribution.

Am I doing something wrong?

$\endgroup$
5
  • 4
    $\begingroup$ You might have become distracted from your original purpose. It sounds like you want to compare two or more groups in a large dataset, using the Kruskal-Wallis test. If the difficulty is that your R code will not handle more than 5000 observations, just use the built-in kruskal.test: it has no such limitation. For example, n <- 5e4; kruskal.test(rnorm(n), floor(runif(n, 0, 3))) performs this test with 50,000 random data values in three groups. $\endgroup$
    – whuber
    Feb 15 '15 at 21:40
  • 2
    $\begingroup$ The title of your post is a bit misleading as well. The interpretation of the test does not change if you use another statistical package to do the analysis. $\endgroup$ Feb 16 '15 at 0:44
  • 2
    $\begingroup$ In any case, it's perfectly feasible to compute a Kruskal-Wallis statistic directly and compare with the asymptotic distribution, without any need for a specific function or package, as long as you have something that will do the relevant ranking, averaging, squaring, summing and so on. What kruskal wallis test function are you calling? $\endgroup$
    – Glen_b
    Feb 16 '15 at 1:23
  • 3
    $\begingroup$ One more thing: given such a large sample it is no surprise that the null hypothesis is rejected. However, that does not necessarily mean your data isn't close to normal. (If it's the case, you could assume normality and the assumption need not make much harm.) $\endgroup$ Feb 16 '15 at 6:00
  • $\begingroup$ I think you should try jarque bera test if you have more than 10000 data $\endgroup$ Mar 20 '15 at 9:47
3
$\begingroup$

Based on (Hair et al., 1998), when observations are above 1000 the K.S test becomes highly sensitive which means small deviations from normality will result in p values below .05 and thus rejecting the normality. Thus for above 1000 observations it is suggested to use graphical tests as well. Try qqPlot and hist to graphically see if data is normal or not.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.