# What to conclude about the results if the p-value is same in 2 Kolmogorov-Smirnov tests?

I am comparing observed and simulated acceleration distributions. There are 3 data sets:

1) acco = observed accelerations
2) acc15 = simulated accelerations data 1
3) acc16 = simulated accelerations data 2

I did 2 sample Kolmogorov-Smirnov test with acco & acc15, and then acco and acc16. I want to see which of the two simulated acceleration distribution is (more) similar to the observed. I guess it is important to note here that both simulations were calibrated using the observed data using different methods. Here are the test results:

> ks.test(acco$slo_v_m, acc15$saccm)

Two-sample Kolmogorov-Smirnov test

data:  acco$slo_v_m and acc15$saccm
D = 0.23379, p-value < 2.2e-16
alternative hypothesis: two-sided

Warning message:
In ks.test(acco$slo_v_m, acc15$saccm) :
p-value will be approximate in the presence of ties
> ks.test(acco$slo_v_m, acc16$saccm)

Two-sample Kolmogorov-Smirnov test

data:  acco$slo_v_m and acc16$saccm
D = 0.37959, p-value < 2.2e-16
alternative hypothesis: two-sided

Warning message:
In ks.test(acco$slo_v_m, acc16$saccm) :
p-value will be approximate in the presence of ties


You can see that both resulted in same p-value but different D statistic. What can I conclude about these data? Can I say thatacc15 is better than acc16 when compared with acco?

• These p-values tell you neither distribution is anything like the reference distribution. This is somewhat like asking us which is more like a crow: a pencil or a stick of chewing gum? – whuber May 18 '17 at 17:07

These P-values aren't the same, they are just both less than $2.2 \times 10^{-16}$. From this I would conclude:
2. Your software isn't very carefully coded. It's probably computing a right tail probability as 1 minus a left cumulative probability, which means it can't, in normal double-precision arithmetic, compute any P-value less than $\sim 10^{-16}$, and already looses a lot of accuracy for values greater than that.