I'm trying to use KS test to determine whether one group of data is scholastically dominates another. So I'm studying dataset regarding performance of companies, which are divided into 2 groups. Instead of comparing mean values for this two groups, I follow [1] and want to compare distributions using KS test (Table 3). They do two tests: one sided (A less then B) and two sided (equality). For that I use STATA's ksmirnov command, the problem is how to interpret the output. It return D and p but what one can conclude from these values is not clear for me. For instance, for my groups it returns:

. ksmirnov performance, by(myGroup)
Two-sample Kolmogorov-Smirnov test for equality of distribution functions

 Smaller group       D       P-value  Corrected
 0:                  0.0047    0.972
 1:                 -0.1635    0.000
 Combined K-S:       0.1635    0.000      0.000

The 0 is checking hypothesis that group0 has smaller values then group1. The 1 for hypothesis that group0 has larger values then group1. But I do not understand how to interpret D and p. What is the unit of D and is it big enough to accept hypothesis (for instance, for the confidence 0.05)?

[1] http://www.etsg.org/ETSG2012/Programme/Papers/329.pdf

  • 2
    $\begingroup$ $D$ is a difference of cdfs (cdfs give probabilities). The p-value is also a (conditional) probability. Are you unfamiliar with how p-values relate to hypothesis testing? $\endgroup$ – Glen_b -Reinstate Monica Oct 12 '14 at 17:41
  • $\begingroup$ So how is D useful? I mean what kind of interpretation one can give to, for instance, output above? p-values as far as I understood are responsible for acceptance of the H. $\endgroup$ – Kirill Lykov Oct 13 '14 at 12:38
  • 1
    $\begingroup$ D is the measure of how discrepant the two ECDFs are. See the diagram here. The p-value is the probability of obtaining a test statistic result at least as extreme the one observed if the null were true. $\endgroup$ – Glen_b -Reinstate Monica Oct 13 '14 at 12:41
  • $\begingroup$ So having the table above, I conclude that there are many values in group0 which are smaller values then group1 (p-value = 0.972) yet the difference is small. And another hypothesis (group0 has larger values) might be rejected yet there are some extreme cases when the difference is huge (0.16)? $\endgroup$ – Kirill Lykov Oct 13 '14 at 13:18
  • $\begingroup$ The p-value you quote doesn't support the conclusion in your first sentence. You don't reject alternative hypotheses (as in your second sentence). $\endgroup$ – Glen_b -Reinstate Monica Oct 13 '14 at 14:39

The Ds are the test statistics and they derive from the differences between the empirical cumulative distribution functions of the two groups. Therefore, they are the differences of probabilities. The p-values have their normal interpretation: if $pval \leq \alpha$, reject the null hypothesis; where $\alpha$ is a predetermined significance level.

Stata also gives an additional p-value for the non-directional hypothesis (Combined K-S), corrected for small samples.

Examples and details of what Stata does are in [R] ksmirnov, including the math in the Methods and formulas section.

An example of a "manual" computation of the Ds is:

set more off

*------ example data -----

use http://www.stata-press.com/data/r12/ksxmpl

*----- manual computation -----

bysort group: cumul x, gen(cumd)

sort x

gen cumd1 = cumd 
replace cumd1 = cumd1[_n-1] if group != 1

gen cumd2 = cumd 
replace cumd2 = cumd2[_n-1] if group != 2
replace cumd2 = cumd2[2] in 1

gen diff = cumd1 - cumd2

summarize diff, meanonly
display  "" _n ///
         "Results are:" _n ///
         "This is D+ : `r(max)'" _n ///
         "This is D- : `r(min)'"

line cumd1 cumd2 x, sort // graph the cdfs

*----- direct computation -----

ksmirnov x, by(group)

The "manual" approach is from [1], which Stata cites in its manual.

[1] Riffenburgh, R. H. 2005. Statistics in Medicine. 2nd ed. New York: Elsevier.

  • $\begingroup$ Am I right that having the STATA output I have in the post I can conclude that H0 is accepted because p=0.972 and H0 is rejected since p=0? $\endgroup$ – Kirill Lykov Oct 13 '14 at 12:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.