Kolmogorov-Smirnov of two heart rate samples

Question

I'm accumulating a lot of vital signs from emergency department patients, and would like to assess if different features yield significantly different distributions.

Eg. for heart rate I have 888.424 measurements (stored in all_hr) for all patients and 321.357 measurements for geriatric patients (stored in g_hr). Summary stats yield:

summary(all_hr$hr) 
Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
2.00   74.00   87.00   88.56  101.00  242.00 

summary(g_hr$hr)
Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
2.00   72.00   85.00   87.77  100.00  207.00 

When plotting the ecdfs for these samples I get:

I'm trying to comprehend how to use the Kolgomorov-Smirnov ks.test() function in R for this:

ks.test(all_hr$hr,g_hr$hr,alternative="two.sided")

Two-sample Kolmogorov-Smirnov test

data:  all_hr$hr and g_hr$hr 
D = 0.0289, p-value < 2.2e-16
alternative hypothesis: two-sided 

Warning:
In ks.test(all_hr$hr, g_hr$hr, alternative = "two.sided") :
p-values will be approximate in the presence of ties

So as far as I understand this test, the hypothesis that samples are from different distributions is true?

Now, I've also attempted a similar approach, but grouping patients by speciality ward instead. E.g. here is the ecdfs two randomly drawn 6.000 sample distributions for neurological and gastro-internal patients:

Yet, when running ks.test on these two samples, I also get the same result

ks.test(N.vs.hr.rs$hr,S.vs.hr.rs$hr,alternative="two.sided")

Two-sample Kolmogorov-Smirnov test

data:  N.vs.hr.rs$hr and S.vs.hr.rs$hr 
D = 0.6307, p-value < 2.2e-16
alternative hypothesis: two-sided 

Warninf:
In ks.test(N.vs.hr.rs$hr, S.vs.hr.rs$hr, alternative = "two.sided") :
p-values will be approximate in the presence of ties

So, in summary...is this test a valid way of asserting differences in distributions? And should I have no quarrels with the fact that two very dissimilar ecdf plots yield the same ks.test results?

Answer 1

In the first case, are the geriatric patients in both datasets? If so, then that will affect the results, the assumption is that the 2 samples are independent of each other and if you have some patients values in both data sets then that makes them not independent. It would be better to compare everyone other than the geriatric patients to the geriatric patients.

That said, with over 800 thousand observations you will have power to find differences that are very small. Your 2 outputs are not identical, look at the value for D, it is just that with such a high sample size the p-value is essentially 0 (the < 2.2e-16 means that it is extremely small and not worth narrowing it down any further).

So, yes the KS test is valid for finding differences in distributions (if you have the independence), but with such large samples there is a huge difference between statistical significance (what the test reports) and practical or meaningful significance (does the difference matter on a practical level).

Answer 2

The two tests yielded the same result because the function is returning a lower bound for p-value of less than 2.2e-16. The test statistics D is actually different.

Yes, the null is that the two samples are drawn from the same distribution -> p < 2.2e-16 suggests otherwise.

This question belongs in stats.stackexchange