How reliable is using KS test (https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.ks_2samp.html) for change-point detection (in a single vector)? Is there a better way for this, in Python?

  • $\begingroup$ Could you please explain how you conceive of applying the KS test to find a changepoint? The reference you give doesn't address this. $\endgroup$ – whuber Jun 13 '18 at 19:13
  • $\begingroup$ @whuber I go over each point and apply KS on all time-series before and after this point, to test if the distribution differ $\endgroup$ – okuoub Jun 13 '18 at 19:18
  • 2
    $\begingroup$ That seems like it would cause pretty serious multiple testing problems. $\endgroup$ – Matthew Drury Jun 13 '18 at 19:56
  • 1
    $\begingroup$ The multiple testing problems can be handled--this approach is, in spirit, much like many other methods to find changes in level, slope, or variance in a series of data. However, it's not clear how effective it would be. It would likely not be a very powerful method because the KS test is looking for any kind of change rather than a specific change. The idea is intriguing enough to be worth some investigation, but first you ought to make sure this is the right test for whatever you're trying to learn about these data. $\endgroup$ – whuber Jun 13 '18 at 22:27
  • 1
    $\begingroup$ You need to provide details in your post. If your vector is binary, the KS test is inapplicable and much simpler methods are available. The use of clusters of three does not sound consistent with your stated problem. A test of whether one vector has a greater proportion of ones than another is called a binomial test of proportions--a search will take you to hundreds of posts about such tests on this site. $\endgroup$ – whuber Jun 14 '18 at 12:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.