1
$\begingroup$

I have two sets of data and I want to determine whether they come from similar distributions or not so I am using the Kolmogorov-Smirnoff Test.

So I understand that if I get a p-value of less than 0.05 I can say that i'm 95% confident that they come from different distributions?

So my question is if I get a p-value of 0.33 am I 67% confident that they come from different distributions? Is that a correct interpretation?

And conversely, how do I say how confident I am that they come from the same distribution, do I need a different test for that? Or is it just the inverse of above?

Thanks

Improve this question
$\endgroup$
2
  • $\begingroup$ In Russian, the names Kolmogorov and Smirnov end with the same letter and so should end with the same letter when transliterated. The transliteration -off is very old-fashioned; regardless of that there can be no grounds for inconsistency. $\endgroup$
    – Nick Cox
    Mar 29, 2016 at 13:22
  • $\begingroup$ Rather than wanting to be assured of what form of words you might use, which seems more a question of diplomacy, etiquette or liturgy, the key question is surely how far and in what ways your distributions differ. A quantile-quantile plot will tell you immensely more than any such test. $\endgroup$
    – Nick Cox
    Mar 29, 2016 at 13:24

1 Answer 1

Reset to default
1
$\begingroup$

want to determine whether they come from similar distributions

A goodness of fit test doesn't say how similar two distributions are; once you pin down a suitable way to measure similarity (suitable to your particular purpose), that would be more like some kind of effect size.

So I understand that if I get a p-value of less than 0.05 I can say that i'm 95% confident that they come from different distributions?

No. That's not a correct interpretation of what a p-value tells you.

So my question is if I get a p-value of 0.33 am I 67% confident that they come from different distributions? Is that a correct interpretation?

No.

And conversely, how do I say how confident I am that they come from the same distribution,

You can't. (Well, possibly you can make a statement vaguely sort of similar to that if you're a Bayesian, but then you wouldn't be doing a Kolmogorov-Smirnoff test. And it shouldn't contain the word "confidence" since that has a particular meaning in statistics that wouldn't really apply.)

Or is it just the inverse of above?

Nope.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.