# Normalization of data before using a Kolmogorov-Smirnov test

I am working with two sets of data and would like to determine if they come from the same distribution -- hence the two population KS-test. At this point, I have binned both sets of data, counting the number of entries in each bin, to create relative frequency histograms. These binned arrays look like:

dis1 = [0.0, 4.0, 9.0, 10.0, 23.0, 26.0, 43.0, 44.0, 38.0, 4.0]
dis2 = [116.0, 160.0, 209.0, 247.0, 415.0, 455.0, 382.0, 288.0, 161.0, 44.0].


When I perform the KS-test for 2 datasets, I get:

[h,p]=kstest2(dis1,dis2)
h = 1
p = 1.7012e-04


meaning that the null hypothesis was rejected and the samples do not come from the same distribution.

Now, if I normalize the binned data with respect to the maximum in each array, I have:

norm_dis1=[0.0, 0.090909090909090912, 0.20454545454545456, 0.22727272727272727, 0.52272727272727271, 0.59090909090909094, 0.97727272727272729, 1.0, 0.86363636363636365, 0.090909090909090912]
norm_dis2 = [0.25494505494505493, 0.35164835164835168, 0.45934065934065932, 0.54285714285714282, 0.91208791208791207, 1.0, 0.83956043956043958, 0.63296703296703294, 0.35384615384615387, 0.096703296703296707],


such that now the norm_dis arrays vary from [0,1]. The 2 population KS-test for these arrays gives:

[h,p]=kstest2(norm_dis1,norm_dis2)
h = 0
p = 0.3129,


such that the null hypothesis is accepted and the populations come from the same distribution.

These two results are seemingly discrepant. What is the statistical difference between normalizing and not normalizing? And which is the correct statistical analysis of these two populations?

-
Why are you binning the data? This would seem to be unnecessary indeed unhelpful. I'm not sure what software you're using but are you sure kstest2 works the way you think it does? Perhaps it's expecting the actual data, not a set of bins. –  Peter Ellis Apr 17 '12 at 4:51
I'm binning the data because I want to determine if the population histograms are similar; this is rather common in my field. The data itself is not binned - dis1 and dis2 are the number of "things" within each bin. –  cosmosis Apr 17 '12 at 5:05
It seems like the kstest of the binned data is now just telling you that the two distributions have a similar shape- the means of your two original samples are still hugely discrepant, but the spread of scores around those two means is similar (in the sense that the ratio of mean1:mean2 is similar to variance1:variance2) –  Marius Apr 17 '12 at 5:31
@Marius - I'm afraid I don't understand. The kstest is telling me that the distributions are both similar and not similar, depending on whether I normalize or not. –  cosmosis Apr 17 '12 at 6:08
Cosmo, that's because your normalizing changes the locations and scales so that you have two new distributions guaranteed to be approximately alike by virtue of that normalization. But please pay attention to @Peter's comment: the KS test is designed for unbinned data and doesn't really work properly with binned data to begin with. (You don't even give it information about bin counts, which is crucial.) Do not be overly concerned with what's common in your field: it is not in the least unusual for some (apparently) common practices to be statistically inferior--or worse. –  whuber Apr 17 '12 at 6:13