1
$\begingroup$

I have two curves and I want to be able to calculate the probability of these curves coming from different distributions or another appropriate statistic.

Each curve is fitted through the mean of clusters of data (x's and o's in diagram) at differing points on the x-axis, each data cluster is also non-parametrically distributed.

Is there a statistical test that would be appropriate to tell me how likely it is that the data/curves come from different distributions i.e. one is significantly different than the other?

        +   -         x
        |    o        ~
        |    -        x~
        |    o         ~
        |     -        ~~
        |     --        ~~
        |      -         ~
        |      --         ~~~
        |       -            ~~~
        |       -               ~~    x
        |        -o               ~~~ x
        |         -                  ~~ 
        |         o-                  x  ~ 
        |         o-                  x   ~ ~~             x
        |           --                        ~~~ ~~~~~~~~~~~
        |            ----        o                         x
        |                - ----  o                         x
        |                      ----- ----               o
        |                        o       - ---- - -  - ---
        |                                               o
        |
        +----------------------------------------------------------+

I've looked here Comparison of two curves but I believe my problem has some distinct differences

$\endgroup$
  • $\begingroup$ About all you are assuming, implicitly, seems to be single-valued curves (i.e. for each $x$, there is a single mean of $y$). So, I think the main, and possibly only, possibility is that you simulate drawing two groups randomly from your combined data, fit your curves and then do a kind of line-up comparing the "real" pattern and the simulated patterns. If the real pattern is genuine it will stand out from the others. $\endgroup$ – Nick Cox Mar 26 '14 at 11:18
  • $\begingroup$ @Nick Cox Thanks for the answer although I am not sure I fully understand. Are you suggesting I create a simulated combined curve from both data sets and then use some test to see if the actual curves are different. What test would this be? $\endgroup$ – CatsLoveJazz Mar 26 '14 at 11:24
  • $\begingroup$ No; I am suggesting that you simulate drawing two curves repeatedly using the same methods. See stat.wharton.upenn.edu/~buja/PAPERS/… for the flavour. Warnings: I can't see any scope for a plug-in or off-the-shelf test if the question is just "I have two curves: are they genuinely different?" which is no more a precise statistical question than "I have two friends: are they genuinely different?". However, there are newer ideas, as in the paper cited, but they usually require some custom programming. $\endgroup$ – Nick Cox Mar 26 '14 at 11:45
  • $\begingroup$ @Nick Cox I understand now, thank you. I'm certainly no statistician but for the engineering work I do we typically phrase the question something like: "What is the probability that the two sets of data (curves in this case) are due to a genuine difference rather than noise" $\endgroup$ – CatsLoveJazz Mar 26 '14 at 12:02
  • $\begingroup$ I am not a statistician either, but there is no free lunch in statistics. You might have that question but it is very difficult to answer. It is not far from the ultimate vague question "are my data meaningful?". $\endgroup$ – Nick Cox Mar 26 '14 at 12:11
1
$\begingroup$

You might be looking for the two sample K-S test.

Matlab Stats toolbox has an implementation, kstest2:

kstest2(x1,x2) returns a test decision for the null hypothesis that the data in vectors x1 and x2 are from the same continuous distribution, using the two-sample Kolmogorov-Smirnov test. The alternative hypothesis is that x1 and x2 are from different continuous distributions.

$\endgroup$
  • $\begingroup$ I'm unclear exactly how I would use this. Would the vectors be the fitted points from the curve or include the data clusters around the known points? Thanks. $\endgroup$ – CatsLoveJazz Mar 28 '14 at 9:40
  • $\begingroup$ I would use your raw data points as x1 and x2. If kstest2 returns a 1, then the null hypothesis has been rejected, meaning that samples x1 and x2 are likely drawn from different "true" distributions. $\endgroup$ – Steve Osborne Mar 28 '14 at 16:19
  • $\begingroup$ I'm not sure I agree that combining each cluster of data from each curve into a single distribution is necessarily an appropriate way to do the analysis, but I may be wrong. $\endgroup$ – CatsLoveJazz Mar 28 '14 at 16:28
  • 1
    $\begingroup$ Not sure what you mean about "combining" the data ... You've got two sets of data. Each represents samples coming from some underlying distribution. I understood your problem as wanting to test whether those two sets of samples come from the same underlying distribution, which K-S pretty much does. $\endgroup$ – Steve Osborne Mar 28 '14 at 16:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.