Skip to main content
1 of 4
Glen_b
  • 290.5k
  • 37
  • 652
  • 1.1k

1- Is one of these statistics (KDE's p-value, KS statistic or the two-tailed p-value) recommended for my needs? If so, why?

Your needs as expressed do not seem to be sufficiently clearly defined as to differentiate between them. They both test for a difference in distribution.

2- What is the difference between the "KS statistic" and a "two-tailed p-value"?

The two sample Kolmogorov-Smirnov statistic is the largest difference in ECDFs for the two samples:

enter image description here

(The data here is the same data I generated for your other question. Here the A sample is red and the B sample is blue.)

The height difference in ECDFs at x=35 is 1/6 or about 0.1667, the same as the value produced by calculating the statistic:

> ks.test(A,B)

        Two-sample Kolmogorov-Smirnov test

data:  A and B 
D = 0.1667, p-value = 0.6228
alternative hypothesis: two-sided 

The meaning of the p-value is as for any hypothesis test - the probability of obtaining a statistic at least as unusual (in this case, at least as large) if the null hypothesis were true.

3- Will the difference in the number of elements in each set affect the outcome of these statistics?

No, the KS test, and (to my understanding) the KDE-based one both handle different sample sizes.


All that said, I recommend you consider whuber's words most carefully. There's a lot of good advice packed into very few words.

Glen_b
  • 290.5k
  • 37
  • 652
  • 1.1k