0
$\begingroup$

I am comparing observed and simulated acceleration distributions. There are 3 data sets:

1) acco = observed accelerations
2) acc15 = simulated accelerations data 1
3) acc16 = simulated accelerations data 2

I did 2 sample Kolmogorov-Smirnov test with acco & acc15, and then acco and acc16. I want to see which of the two simulated acceleration distribution is (more) similar to the observed. I guess it is important to note here that both simulations were calibrated using the observed data using different methods. Here are the test results:

> ks.test(acco$slo_v_m, acc15$saccm)

    Two-sample Kolmogorov-Smirnov test

data:  acco$slo_v_m and acc15$saccm
D = 0.23379, p-value < 2.2e-16
alternative hypothesis: two-sided

Warning message:
In ks.test(acco$slo_v_m, acc15$saccm) :
  p-value will be approximate in the presence of ties
> ks.test(acco$slo_v_m, acc16$saccm)

    Two-sample Kolmogorov-Smirnov test

data:  acco$slo_v_m and acc16$saccm
D = 0.37959, p-value < 2.2e-16
alternative hypothesis: two-sided

Warning message:
In ks.test(acco$slo_v_m, acc16$saccm) :
  p-value will be approximate in the presence of ties  

You can see that both resulted in same p-value but different D statistic. What can I conclude about these data? Can I say thatacc15 is better than acc16 when compared with acco?

$\endgroup$
  • 3
    $\begingroup$ These p-values tell you neither distribution is anything like the reference distribution. This is somewhat like asking us which is more like a crow: a pencil or a stick of chewing gum? $\endgroup$ – whuber May 18 '17 at 17:07
2
$\begingroup$

These P-values aren't the same, they are just both less than $2.2 \times 10^{-16}$. From this I would conclude:

  1. As whuber said, neither acc15 nor acc16 is a good fit to acco.

  2. Your software isn't very carefully coded. It's probably computing a right tail probability as 1 minus a left cumulative probability, which means it can't, in normal double-precision arithmetic, compute any P-value less than $\sim 10^{-16}$, and already looses a lot of accuracy for values greater than that.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.