# Covariance between two binomial random variables or expectation of product of binomial random variables

I have an empirical distribution $$S_n(x)$$ (= proportion of samples less than equal to x) from a random sample $$X_1, X_2, ..., X_n$$ for a random variable $$X \sim F_X$$. Consider the random variable $$T_n(x) := n.S_n(x)$$. This is a binomial random variable $$B(n, F_X(x))$$.

What is the expectation $$E(T_n(x)T_n(y))$$ when $$x < y$$?

Reference: Corollary 2.3.13, Nonparametric Statistical Inference by Gibbons-Chakraborti, Fifth edition.

They have given the answer as $$nF_X(x) + n(n-1)F_X(x)F_X(y)$$.

I have tried to write it as a product of indicator variables, $$E(I(X_i \le x)I(X_i \le y))$$ but I'm unable to complete the derivation. I appreciate hints.

Edit :

With @whuber's hint, I separate the expectation into two cases, when $$i = j$$ and when $$i \ne j$$:

$$E(T_n(x)T_n(y)) = E(I(X_i \le x)I(X_j \le y))$$

$$E(T_n(x)T_n(y)) = E(I(X_i \le x)I(X_i \le y)) + E(I(X_i \le x)I(X_j \le y))$$

The product is one if and only if both the indicators are one.

In the first term, as $$x < y$$, $$I(X_i \le x)I(X_i \le y) = I(X_i \le x)$$ as the latter indicator is a superset of the former. This has an expectation $$nF_X(x)$$.

In the second term, as the samples are independent, I can write the expectation as product of expectations, $$E(I(X_i \le x)I(X_j \le y)) = E(I(X_i \le x))E(I(X_j \le y))$$.

Now, the first expectation is similar to the first case. But, why is the second expectation $$(n-1)F_X(y)$$?

• Clearly -- correcting a typo -- you need to compute $E[I(X_i\le x)I(X_j\le y)].$ Because the sample is random, the $X_i$ are exchangeable, implying there are just two such expectations to compute: one where $i=j$ and the other where $i\ne j.$ Where do you run into problems computing them?
– whuber
Commented Sep 2, 2023 at 16:02
• @whuber Thanks a lot! I see my mistake, will try with this. Commented Sep 2, 2023 at 16:16

The basis of this result is the observation that

$$F(x) = \Pr(X_i\le x) = E[I(X_i\le x)]$$

for all $$i.$$

For $$i\ne j,$$ the variables $$X_i$$ and $$X_j$$ are independent, whence so are their indicators, giving

$$E[I(X_i\le x)I(X_j\le y)] = E[I(X_i\le x)]E[I(X_j\le y)] = F(x)F(y).$$

When $$i=j,$$ because $$x\lt y$$ implies $$I(X_i\le x)I(X_i\le y)= I(X_i\le \min(x,y))=I(X_i\le x),$$ it follows

$$E[I(X_i\le x)I(X_i\le y)] = E[I(X_i\le x)] = F(x).$$

Consequently, because $$T_n(x) = I(X_1\le x)+I(X_2\le x)+\cdots + I(X_n\le x)$$ counts the number of $$X_i$$ less than or equal to $$x,$$ linearity of expectation gives

\begin{aligned} E[T_n(x)T_n(y)]&=E\left[\sum_{i=1}^n \sum_{j=1}^n I(X_i\le x)I(X_j\le y)\right]\\ &= \sum_{i=1}^n E[I(X_i \le x)I(X_i\le x)] + \sum_{i\ne j}^n E[I(X_i\le x)I(X_j\le x)]\\ &= \sum_{i=1}^n F(x) + \sum_{i\ne j}F(x)F(y)\\ &= nF(x) + n(n-1)F(x)F(y). \end{aligned}

The coefficient $$n$$ counts the number of terms in the double sum where $$i=j$$ while the coefficient $$n(n-1)=n^2-n$$ counts the remaining terms in the double sum where $$i\ne j.$$