I am trying to test whether two distributions are the same or not. I am assuming that I cannot observe the number of realized draws from this distributions but only the empirical estimates of the two distributions. However, it is possible to resample these distributions I get out of a formula with a resample of the data that allows me to calculate the distributions. I believe the Kolmogoroff-Smirnov test is not applicable. However, I want to focus on a similar test statistic

$$D = max_{x} (|\hat{F}_{1}(x) - \hat{F}_{2}(x)|)$$

Under the null, both distributions should be the same. Now if I got it right, I would have to proceed in the following to nonparametrically bootstrap the test statistic's distribution

Resample my data, calculate the distributions for the new sample and calculate the test statistic for it, subtract the initial test statistic and repeat this $n$ times. Then, look for the $95^{th}$ percentile of this distribution and if the initial test statistic is larger, reject the null.

Am I totally off or is this somehow consistent with theory?

  • $\begingroup$ Good question. Subtracting the initial test statistic might not be 100% correct. Technically, one would need to modify one of the two samples to make the original test statistic 0. But this cannot be done without a lot of arbitraryness. $\endgroup$ – Michael M Apr 30 '18 at 13:24
  • $\begingroup$ thank you for your comment Michael M. I wonder whether this would imply that I have to calculate some moments of my data and try to subtract it from the original sample to make the initial distributions as close as possible? But somehow I feel that there is something wrong with this approach. $\endgroup$ – saguru Apr 30 '18 at 13:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.