# Are these estimators of $P(X<Y)$ asymptotically normally distributed?

I have two continuous random variables, $X$ and $Y$, whose distributions are unknown. I can draw samples from them and in particular, I am interested in estimating $P(X<Y)$ based on those samples. Let's assume that each sample has $n$ elements: $x_1, x_2, \ldots, x_n$ are drawn from $X$ and $y_1, y_2, \ldots, y_n$ from $Y$.

Can I use $$T=\frac{1}{n^2}\sum_{i=1}^{n}\sum_{j=1}^{n}I(x_i < y_j)$$ as the estimate of $P(X<Y)$ if I assume $X$ and $Y$ are independent and is such an estimator asymptotically normally distributed? If I knew

Also, if I assume that $X$ and $Y$ are dependent, can I use $$S=\frac{1}{n}\sum_{i=1}^{n}I(x_i < y_i)$$ as the estimator of said probability and is it asymptotically normally distributed?

I'm asking about asymptotic normality because I would like to determine lower and upper bounds of $P(X<Y)$ and think that confidence intervals can act as such.

P.S. $I(E)=1$ if $E$ is true, and $0$ otherwise.

• You can apply the Central Limit Theorem directly to $S$ because it's the sum of bounded iid variables. (Indeed, it is proportional to a Binomial$(n, \Pr(X\lt Y))$ distribution.) It takes a little bit more work to apply it to $T$ directly because the variables, although identically distributed and bounded, are correlated. However, you can compute their variance-covariance matrix explicitly, which gives one way to draw the conclusion (as well as figure out the bounds). Of greatest interest, perhaps, is the question whether $T$ is any better an estimator than $S$. – whuber Oct 30 '17 at 17:27
• @whuber Thanks for the comment. If I may ask a follow-up question, how would the variance-covariance matrix help me figure out the bounds? If you can point me out to some good tutorials or excerpts from books, I'd be very grateful. :) – Milos Oct 30 '17 at 17:58
• All you need to know about any Normal distribution is its mean and variance; the bounds you refer to will depend on the variance. The variance-covariance matrix of the $n^2$ variables $I(x_i\lt y_j)$ determines the variance of $T$. When $n$ is large enough to make $T$ approximately Normal, then you have all the information you need. – whuber Oct 30 '17 at 18:03

The statistic $T$ is an example of a general $U$-statistic, introduced by Hoeffding in his 1948 paper A class of statistics with asymptotically normal distribution. Moreover, it is among the most famous of that class, namely the Mann-Whitney U test. It has lower mean squared error than $S$, for it equals $E\left(S\mid X_{\left(1\right)},X_{\left(2\right)},\ldots,Y_{\left(1\right)},Y_{\left(2\right)},\ldots\right)$, and since the order statistics are sufficient, the Rao-Blackwell theorem can be applied. Furthermore, it is normally distributed, a fact that follows from a general theorem on $U$-statistics, see the chapters on $U$-statistics in e.g. Lehmann's Elements of Large Sample Theory or Serfling's Approximation Theorems of Mathematical Statistics. Its limiting distribution, with a small sample correction, can be found on wikipedia., and it is implemented in R through the function $\mathtt{wilcox.test}$ (with the option $\mathtt{paired=FALSE}$).
Note that the definition of the Mann-Whitney U test is ambiguous. Sometimes it is defined as the $T$ above, but sometimes it counts "victories for $x_i$" as positive and "victories for $y_i$" as negative. This is the approach taken in e.g. the R function $\mathtt{wilcox.test}$.