1
$\begingroup$

The setup of my problem is that I have some response variable, $Y$, and a predictor, $X$. I have measurements on both variables from two groups. In each group, there is one $Y$ per $X$. I want to compare the curves in $\mathbb{R}^2$ and assess if they were generated by the same process. An idea that has been floated past me is to use the Kolmogorov-Smirnov test. I do not see this problem as a univariate comparison between groups, so KS, to me, is invalid. However, the idea of measuring the maximum distance between curves appeals to me.

In studying the nitty-gritty of the KS test, I see that the p-value comes from comparing the maximum vertical distance between CDFs to a distribution related to a Brownian bridge between the CDFs (something like that). This idea makes sense to me in more generality. If any two curves are generated by some process, sampling will cause the curves not to be the same, but we can say if the lack of similarity is likely to occur if the two curves were generated by the same process; then the standard KS test is a special case.

My Questions

1) Has this idea been worked out in the generality I describe?

2) Is there a software implementation of this idea?

I have found this thread that seems to address my question about comparing curves, and while the dynamic time warping method may be superior, I want to pursue an approach like KS.

Thanks!

PS: The curves are assured to be non-increasing, if that matters.

Improve this question
$\endgroup$
5
  • $\begingroup$ In most applications this is a poor idea because the test statistic is exquisitely sensitive to random variation in the response: it is, in effect, the maximum among the errors. The situation becomes almost impossible when the groups don't share exactly the same values of the regressors, because then everything depends on how you fit the curves. Thus, it sounds like your responses are paired and you're trying to assess the spread of the univariate dataset of their differences--something that t-tests, etc. are well suited for. $\endgroup$
    – whuber ♦
    Commented Nov 27, 2019 at 19:45
  • $\begingroup$ @whuber I do not think that I am assured of having totally paired data. If I do have pairs, however, then comparing to $x=0$ with some kind of Brownian motion intrigues me. Now why do you say that the test statistic is so sensitive to random variation in the response? Maybe this is an issue for KS and is a good reason not to use KS (despite its popularity), but wouldn't that be an issue for KS? PS: The curves are assured to be non-increasing, if that matters. $\endgroup$
    – Dave
    Commented Nov 27, 2019 at 19:53
  • 3
    $\begingroup$ It's not an issue for KS because the "curves" it is comparing are empirical distribution functions. That's a severe restriction which, among other things, imposes strong positive correlation among the values. The fact that two completely different situations can be visualized in the same way--in this case, as graphs of functions--has little bearing on whether a statistical procedure applicable to one situation has any useful properties at all in the other. The lesson is that you need to focus on the underlying probability model. $\endgroup$
    – whuber ♦
    Commented Nov 27, 2019 at 20:01
  • $\begingroup$ @whuber I also have strong correlation between my two curves. Perhaps that doesn’t matter unless they actually are CDFs. (They definitely are not.) I am curious to read the details of what’s going on with KS that allows us to compare CDFs. Do you have a reference? $\endgroup$
    – Dave
    Commented Nov 28, 2019 at 16:44
  • $\begingroup$ en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test $\endgroup$
    – whuber ♦
    Commented Nov 29, 2019 at 17:11

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.