Let $X$ and $Y$ be two real random variables with cumulative distribution functions $F_X$ and $F_Y$, respectively.

Define $d = \sup(F_X(x) - F_Y(x))$. If $\hat F_X$ and $\hat F_Y$ are empirical versions of $F_X$ and $F_Y$ respectively, then an estimator of $d$ is $\hat d = \sup(\hat F_X(x) - \hat F_Y(x))$

Does anyone know how to build a confidence interval for $d$ based on $\hat d$?

  • 4
    $\begingroup$ This looks like the test statistic of the Kolmogorov Smirnov test for the equality of two distributions . So check the derivation. The confidence interval will be the acceptance region of the test. $\endgroup$ – Placidia Sep 15 '17 at 22:01
  • 1
    $\begingroup$ K-S acceptance region is constructed under the assumption that $F_X = F_Y$. I don't know how to take into consideration the possible inequality of the cdf's $\endgroup$ – Mur1lo Sep 15 '17 at 22:53
  • $\begingroup$ Are you looking for one number to quantify the distance between two CDFs or a whole confidence region where you allow for some uncertainty at every point along the CDF? $\endgroup$ – Dave Jul 30 at 9:53

You can construct a confidence interval by bootstrap sampling

First sample from your empirical distributions $X$ and $Y$ to generate populations of CDFs $\left\{(\hat F_X)_1,\dots,(\hat F_X)_n\right\}$ and $\left\{(\hat F_Y)_1,\dots,(\hat F_Y)_n\right\}$

Next sample pairs of these CDFs to compute a population of bootstrap estimates of $\hat d$: $\left\{\hat d_1,\dots,\hat d_m\right\} = \left\{\sup\left((\hat F_X)_{i_1}-(\hat F_Y)_{j_1}\right),\dots,\sup\left((\hat F_X)_{i_m}-(\hat F_Y)_{j_m}\right)\right\}$

Then compute the differences between these bootstrap estimates $\hat d_*$ and the true $\hat d$: $\{\delta_1,\dots,\delta_m\} = \{\hat d_1 - \hat d,\dots,\hat d_m - \hat d\}$

And finally construct your $\alpha$ confidence interval: $\left[\hat d-\delta_{\lceil m\cdot(1-\alpha/2)\rceil}, \hat d-\delta_{\lfloor m\cdot\alpha /2 \rfloor }\right]$

This construction is equivalent to the pivot confidence interval

Note that you will need to make your numbers of samples $n$ and $m$ sufficiently large for this confidence interval to be reliable.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.