I'm creating PDFs of rainfall observations, and I've got six years of data. I'm trying to see if small changes in some model parameters create PDFs that are significantly different than what I'd expect from interannual variability, which I'm representing by the six years of observations. I'm thinking of calculating the distance between the distributions, with perhaps an emphasis on "extreme values", defined to be greater than, say, 50 mm/day and seeing if the distance between two model PDFs (with two different sets of parameters) is greater than, say, the maximum distance between the observational PDFs. I've thought about using the 2-sample Kolmogorov-Smirnov test, but the test says that basically all of the six years are different from each other, and also I'm not sure if the yes/no nature of a hypothesis test is what I want to do here..I want more of a number. Also, given the nature of extreme rainfall, some years have (very rare) values that go out to about 2000 mm/day, while others only go to about 1000 mm/day..not sure if that will impact the metric used.

In the image the bold brown and magenta lines denote regions (southeast and west), but one could imagine that they arise from the two sets of model parameters, and that the spread between them is greater than that between the observations. What's a good way to quantify this difference?

enter image description here

  • 1
    $\begingroup$ The KS test does give you a number: D, the maximum separation of the two EDFs. By the way, since years are cyclical, you should be using the Kuiper test, which is the cyclical variant of KS, and will be invariant to your choice of January 1. $\endgroup$ – David Wright Jun 12 '17 at 19:02
  • $\begingroup$ @David ... but it doesn't emphasize the tail... $\endgroup$ – Glen_b Jun 13 '17 at 2:01
  • $\begingroup$ @DavidWright Yeah the KS test statistic looks at a CDF basically, right? So it could be heavily influenced by the lower rainfall values. $\endgroup$ – Alex Charn Jun 13 '17 at 17:52
  • $\begingroup$ @Glen_b: That's absolutely true. I've gotta say, though, I see the "okay, let's just look at the tails" approach as a highly suspect response to not getting the answer he wanted. If Alex is legitimately only interested changes in some well-defined tail part of the distribution, then he can use the 2-sample test on the truncated distributions (values > some cutoff). Just decide on the cutoff beforehand, Alex; it's not legitimate to change it to get the answer you want. $\endgroup$ – David Wright Jun 13 '17 at 23:25
  • $\begingroup$ @DavidWright So it's a valid and legal use of the 2-sample KS test to only look at values and parts of the PDF greater than some cutoff? $\endgroup$ – Alex Charn Jun 13 '17 at 23:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.