In the following plot you see my empirical data (black) plotted against a hypothesised distribution (blue).

enter image description here

However, a KS-test shows that there is no indication that my sample follows this distribution. Same for the gamma test described by Villasenor and Gonzalez-Estrada (2015).

My question is related to how to explain the mismatch in an easy to understand way (for a report I'm writing). The way I see it, the "mismatch" happens mostly at the peak of the curve. My guess is that since the majority of values lie around this peak, the "penalty" calculated by the KS-test is weighted and is therefore much worse in that area than in other parts of the distribution. If that's true it explains why the minor difference in the peak has such a huge impact even though the rest seems to fit well. Is this reasoning correct? Or is the test not weighted and is there simply too much difference with the distribution overall.

To ask my question in a different way, does a mismatch around the peak density affect the outcome of a KS-test more than a mismatch in areas of lower density?

I am very new to statistic so please forgive my choice of words and limited knowledge. Any help/pointers would be appreciated.

EDIT: following the advice of Nick Cox below, I have created some Q-Q plots. Interestingly the gamma distribution was a really bad fit so I tried some alternative distributions.

enter image description here

Out of these distributions the Q-Q plot for the lnorm distributions seemed the most promising.

enter image description here

However, there is still no statistically significant fit (not even close) and I'm still not sure how to interpret this.

  • $\begingroup$ I think what's key here is that you are trying to think about how K-S works by looking at density functions. But it doesn't know or care about density functions: it's distribution functions that matter. So you need to plot the distribution functions to see where they differ most. More generally, I find these tests fairly useless. With your sample size, even minute discrepancies count as significant at conventional levels. Also, knowing that the fit isn't perfect doesn't take you in any useful direction unless you have other candidate distributions and can check whether they fit better or not. $\endgroup$ – Nick Cox Feb 15 '19 at 9:49
  • $\begingroup$ There is one kind of plot that beats all others hands down in this territory, the quantile-quantile plot. $\endgroup$ – Nick Cox Feb 15 '19 at 9:49
  • $\begingroup$ A detail important here is whether your data are necessarily all positive and if so whether the theoretical distribution follows suit. Also, kernel density estimation doesn't work especially well when the data is highly skewed. Default methods can shunt a lot of the mass into impossible regions. It's often more informative to work on a transformed scale. $\endgroup$ – Nick Cox Feb 15 '19 at 9:53
  • $\begingroup$ @NickCox thank you for the feedback. I'm afraid to say most of your explanation goes over my head, but I tried Q-Q plots and updated my question. Based on Q-Q plots the lnorm seems to be a better fit, but also far from statistically relevant. My data is indeed all positive. $\endgroup$ – peanutman Feb 15 '19 at 10:54
  • $\begingroup$ One point is shown by your density curves. The empirical density estimate doesn't know that the variable can't be negative and shunts some mass below zero. In turn I don't know what "statistically relevant" means. I would fit lognormal, Weibull, gamma and then use log scales on the quantile-quantile plots. $\endgroup$ – Nick Cox Feb 15 '19 at 13:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.