34
$\begingroup$

I have Two samples that I want to test (using python) if they are drawn from the same distribution. To do that I use the statistical function ks_2samp from scipy.stats. It returns 2 values and I find difficulties how to interpret them. Help please!

$\endgroup$
26
$\begingroup$

As Stijn pointed out, the k-s test returns a D statistic and a p-value corresponding to the D statistic. The D statistic is the absolute max distance (supremum) between the CDFs of the two samples. The closer this number is to 0 the more likely it is that the two samples were drawn from the same distribution. Check out the Wikipedia page for the k-s test. It provides a good explanation: https://en.m.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test

The p-value returned by the k-s test has the same interpretation as other p-values. You reject the null hypothesis that the two samples were drawn from the same distribution if the p-value is less than your significance level. You can find tables online for the conversion of the D statistic into a p-value if you are interested in the procedure.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you for your answer. In fact, I know the meaning of the 2 values D and P-value but I can't see the relation between them. How can I define the significance level? Can you give me a link for the conversion of the D statistic into a p-value? $\endgroup$ – meri May 2 '13 at 13:22
  • $\begingroup$ Sure, table for converting D stat to p-value: soest.hawaii.edu/wessel/courses/gg313/Critical_KS.pdf $\endgroup$ – CrossValidatedTrading May 2 '13 at 16:19
  • $\begingroup$ @CrossValidatedTrading: Your link to the D-stat-to-p-value table is now 404. $\endgroup$ – james.garriss Dec 4 '15 at 17:49
  • $\begingroup$ @CrossValidatedTrading Should there be a relationship between the p-values and the D-values from the 2-sided KS test? In some instances, I've seen a proportional relationship, where the D-statistic increases with the p-value. That seems like it would be the opposite: that two curves with a greater difference (larger D-statistic), would be more significantly different (low p-value)... $\endgroup$ – Thomas Matthew Nov 29 '16 at 0:47
  • $\begingroup$ if the p value is > 0.05, then your two samples should be identical and balanced. $\endgroup$ – user798719 Dec 17 '16 at 7:55
5
$\begingroup$

When doing a Google search for ks_2samp, the first hit is this website. On it, you can see the function specification:

This is a two-sided test for the null hypothesis that 2 independent samples are drawn from the same continuous distribution.

Parameters : 
  a, b : sequence of 1-D ndarrays
  two arrays of sample observations assumed to be drawn from a continuous distribution, sample sizes can be different

Returns :   
  D : float,  KS statistic
  p-value : float, two-tailed p-value
| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ The a and b parameters are my sequence of data or I should calculate the CDFs to use ks_2samp? $\endgroup$ – meri May 2 '13 at 13:41
  • $\begingroup$ @meri: there's an example on the page I linked to. $\endgroup$ – Stijn May 2 '13 at 13:54

Not the answer you're looking for? Browse other questions tagged or ask your own question.