Prove that $ \mathbb{E}[XY] - \mathbb{E}[X] \mathbb{E}[Y] = \int_{- \infty}^\infty \int_{- \infty}^\infty (F(x,y)-F_X(x) F_Y(y)) \,dx\,dy,$

Question

I'm studying associated variables now and in one proof the following Lemma is used:

Let $X$ and $Y$ be random variables with distribution $F_X$ and $F_Y$ and joint distribution $F$. Then \begin{align} & \mathbb{E}[XY] - \mathbb{E}[X] \mathbb{E}[Y] \\[8pt] = {} & \int_{- \infty}^\infty \int_{- \infty}^\infty (F(x,y)-F_X(x) F_Y(y)) \,dx\,dy, \end{align} if $\mathbb{E}[XY]$, $\mathbb{E}[X]$ and $\mathbb{E}[Y]$ exist.

The proof begins with introducing to vectors of independent variables:

Let $(X_1,Y_1)$ and $(X_2,Y_2)$ be independent, each distributed to $F$.

Define $I(u,x)=\begin{cases}1& \text{if } u \leq x \\ 0 & \text{otherwise} \end{cases}$.

Then it is said, that $$ 2 (\mathbb{E}[X_1Y_1] - \mathbb{E}[X_1] \mathbb{E}[Y_1]) = \mathbb{E}[(X_1 - X_2)(Y_1-Y_2)] $$ $$ = \mathbb{E} \left[\int_{- \infty}^\infty \int_{- \infty}^\infty [I(u,X_1)-I(u,X_2)][I(v,Y_1)-I(v,Y_2)] \, du \, dv \right].$$ The first equation is clear for me, but I don't get the second one. Some help would be nice here.

Furthermore the proof ends with the argument, that we can take the expectation under the integral sign since the expectation is finite, thats clear. Then we are ready, but I don't get the right result by calculating.To make the proof fit, we would have to end up at 2 times the right handside of the statement.

What I also observed is that this inequality gives a simple proof for $\mathbb{E}[XY] \geq \mathbb{E}[X] \mathbb{E}[Y]$.

Answer 1

For the second equality, think about the integral $$\int_{- \infty}^{\infty} \int_{- \infty}^{\infty} [I(u,X_1)-I(u,X_2)][I(v,Y_1)-I(v,Y_2)]du dv $$ which is a random variable. The structure of the integrand allows the integral to be rewritten as $$ \int_{- \infty}^{\infty} [I(u,X_1)-I(u,X_2)]du \int_{- \infty}^{\infty}[I(v,Y_1)-I(v,Y_2)]dv $$

Consider the integrand involving $u$. Its value is either $1$, $0$ or $-1$ depending on how $u$ relates to $X_1$ and $X_2$. By considering when these three possibilities occur, you should be able to convince yourself that the $u$ integral is just $X_1-X_2$. Similarly for the $v$ integral.

The later section of the proof hinges on evaluating $$\mathbb{E}[[I(u,X_1)-I(u,X_2)][I(v,Y_1)-I(v,Y_2)]]$$ This expectation can be written as a sum of four terms. One such term is $\mathbb{E}[I(u,X_1)I(v,Y_1)]$. Using standard indicator function notation, this is simply $$\mathbb{E}[1_{u\leq X_1}1_{v\leq Y_1}]=P(u\leq X_1,v\leq Y_1)=1-F_Y (v)-F_X(u)+F(u,v)$$ (best seen by drawing a diagram of $u$,$v$ space). Similar calculations with the other terms should give you the desired result.

Note: the final equation assumes that all the distribution functions involved are continuous. The equation may not be true at discontinuity points of $F_Y$, $F_X$. However, the set of such points is countable so the value of the double integral will not be affected.