2
$\begingroup$

I have a set of medical data which contains two samples taken in different time periods. Although the samples do not strictly contain the same people in each group, a large proportion in each group are the same.

My question, is that I have thus far computed a ks.test in R to compare the distributions of the two groups to determine if they're significantly different. This test was used since the data are not normally distributed and I wanted to check for more than just mean differences, but also look at the variance and shape of the distribution which is a result of the Kolmogorov Smirnoff test (as opposed to the t test).

The result from this test was that the distributions are significantly different. Therefore, my question is, if I were to then filter for all of the people who appear in both groups and conduct a paired t test on the differences (which are normally distributed), would it automatically follow that the difference is significantly different to zero? In other words, is there a rule which says that if the distributions of the two groups are significantly different (using the sample of people who appear in both groups), then this must mean that the per person differences are significantly different to 0?

I understand that this wouldn't work the other way, i.e. if the paired t test was significant, then this wouldn't necessarily mean that the KS test would be significant.

Is that correct?

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ The ks test assumes independence. If you have the same people in both groups, the assumptions of the test don't hold. $\endgroup$
    – Glen_b
    Oct 31, 2018 at 9:38
  • $\begingroup$ I suppose I was generalising such that if the two samples are large enough (i.e. each over 50% of the population), then independence could not be guaranteed.. a push, I know. $\endgroup$
    – sym246
    Oct 31, 2018 at 10:29
  • $\begingroup$ the overlapping/nonoverlapping-samples part makes the question very weird/difficult (statistically speaking). The answer to "does significantly different distribution as measured by KS test -> significantly different means as measured by paired t-test" is no; consider two samples with the same mean but very different variances ... $\endgroup$
    – Ben Bolker
    Oct 31, 2018 at 12:26

1 Answer 1

Reset to default
0
$\begingroup$

No.

It should be fairly straightforward to construct a counterexample. Imagine that the people in both groups experience no change. Then the paired t-test shows no difference. However, the people in the first group but not the second tend to be on the low end, while it is the opposite in the second group. Here is some code simulating what I have in mind.

set.seed(1)
x0 <- rnorm(75,0,1) # common subjects
x1 <- rnorm(25,-2,1)
x2 <- rnorm(25,2,1)
A <- c(x0,x1)
B <- c(x0,x2)
ks.test(A,B)
t.test(x0-x0)

(My R is being weird, so I did not get to run this, but even if it has a mistake, you get the idea.)

As the comments say, however, this is in violation of an assumption of the KS test.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.