3
$\begingroup$

I have multi-sample data which comprises distances between successive points on a line (1D vector), to which a gamma-distribution is fitted (and maximum likelihood parameters obtained). I would like to assess the quality of this fit.

I've read that GoF-tests such as the one-sample Kolmogorov-Smirnov (KS) or Anderson-Darling (AD) are inapplicable in this case and will produce inaccurate p-values because the theoretical distribution being compared to has been derived from the data itself and thus is not independent. Could anybody explain this further as i'm not sure I quite understand why it would be an issue?

Secondly, is bootstrap resampling (iterative sampling with replacement) a viable solution in this case to obtain more-accurate p-values and assessment of the fit? Following the resampling process and obtaining of, for example, KS-stats for each sample - how would one then "average" the results into a final conclusion/assessment or interpret the outcome?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

The standard KS test is built with an assumption that you know the underlying distribution. For instance, you are testing for the standard normal distribution with mean zero and variance one. See item #3 here. For instance, if you assume Student t-distribution and estimate its parameters from the data, then you can't use KS test as it is usually implemented. There's a degree of freedom parameter, which is a problem.

One way to overcome this limitation is by bootstrapping. It's described in this paper: Jogesh Babu, G., and C. R. Rao. "Goodness-of-fit tests when parameters are estimated." Sankhya: The Indian Journal of Statistics 66 (2004): 63-74 They explain how exactly to do it, and why it works for parametric distributions

Improve this answer
$\endgroup$
4
  • $\begingroup$ +1 informative and helpful answer. I will read this paper ASAP although may struggle with the equations/math. In practical terms, would a parametric bootstrap simply consist of obtaining MLE distribution parameters for the actual data and then randomly sampling (with replacement) off of the distribution fit? What is the main purpose of this? What would generally be the next step? $\endgroup$
    – AnnaSchumann
    Oct 31, 2016 at 18:13
  • 1
    $\begingroup$ take a look at this answer with code. there are some other good code examples out there for Lilliefors approach, which I used $\endgroup$
    – Aksakal
    Oct 31, 2016 at 19:32
  • $\begingroup$ Great, thank you! So the bootstrapped KS-stat would be compared to the newly generated critical values rather than the standard ones? $\endgroup$
    – AnnaSchumann
    Nov 1, 2016 at 12:32
  • $\begingroup$ The ks statistic is the same. You bootstrap to get new critical values. $\endgroup$
    – Aksakal
    Nov 1, 2016 at 12:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.