I am trying to compare two data vectors to find out if they come from the same distribution. I computed both K-S test results as well as DKW bound. I uses kstest2 function of MATLAB (with default alpha value of 0.05). This gives back the KS2 Statistic (ks2stat, in short), as well as the decision. In K-S test, the null hypothesis is the two data vectors come from the same distribution; alternative being they do not.
Next, I computed the DKW bound from the ECDF of each data vector with alpha = 0.05. Let's call these DKW1 and DKW2 respectively. As explained here (Section D), I should add the DKW bounds of both ECDFs and then compare it with the K-S distance (which is ks2stat as obtained in the first step).
DKW1 = sqrt(ln(2/alpha)/(2*length(datavector1))); DKW2 = sqrt(ln(2/alpha)/(2*length(datavector2))); DKW = DKW1 + DKW2;
Then, as per the previous link, if ks2stat > DKW, then the two data vectors do NOT come from the same distribution. Otherwise, either the two data vectors do come from the same distribution or the number of samples are not large enough to make any decision.
I noticed that there were quite a few cases when KS2 test gave H1 (alternative) hypothesis as the decision while ks2stat turned out to be less than DKW. Does this mean KS2 test and DKW-based test are testing different hypotheses? Could someone please explain? Any help would be greatly appreciated. Thanks.