If $X$ and $Y$ are two random variables that can only take two possible states, how can I show that $Cov(X,Y) = 0$ implies independence? This kind of goes against what I learned back in the day that $Cov(X,Y) = 0$ does not imply independence...
The hint says to start with $1$ and $0$ as the possible states and generalize from there. And I can do that and show $E(XY) = E(X)E(Y)$, but this doesn't imply independence???
Kind of confused how to do this mathematically I guess.
For binary variables their expected value equals the probability that they are equal to one. Therefore,
$$ E(XY) = P(XY = 1) = P(X=1 \cap Y=1) \\ E(X) = P(X=1) \\ E(Y) = P(Y=1) \\ $$
If the two have zero covariance this means $E(XY) = E(X)E(Y)$, which means
$$ P(X=1 \cap Y=1) = P(X=1) \cdot P(Y=1) $$
It is trivial to see all other joint probabilities multiply as well, using the basic rules about independent events (i.e. if $A$ and $B$ are independent then their complements are independent, etc.), which means the joint mass function factorizes, which is the definition of two random variables being independent.
Both correlation and covariance measure linear association between two given variables and it has no obligation to detect any other form of association else.
So those two variables might be associated in several other non-linear ways and covariance (and, therefore, correlation) could not distinguish from independent case.
As a very didactic, artificial and non realistic example, one can consider $X$ such that $P(X=x)=1/3$ for $x=−1,0,1$ and also consider $Y=X^2$. Notice that they are not only associated, but one is a function of the other. Nonetheless, their covariance is 0, for their association is orthogonal to the association that covariance can detect.
EDIT
Indeed, as indicated by @whuber, the above original answer was actually a comment on how the assertion is not universally true if both variables were not necessarily dichotomous. My bad!
So let's math up. (The local equivalent of Barney Stinson's "Suit up!")
If both $X$ and $Y$ were dichotomous, then you can assume, without loss of generality, that both assume only the values $0$ and $1$ with arbitrary probabilities $p$, $q$ and $r$ given by $$ \begin{align*} P(X=1) = p \in [0,1] \\ P(Y=1) = q \in [0,1] \\ P(X=1,Y=1) = r \in [0,1], \end{align*} $$ which characterize completely the joint distribution of $X$ and $Y$. Taking on @DilipSarwate's hint, notice that those three values are enough to determine the joint distribution of $(X,Y)$, since $$ \begin{align*} P(X=0,Y=1) &= P(Y=1) - P(X=1,Y=1) = q - r\\ P(X=1,Y=0) &= P(X=1) - P(X=1,Y=1) = p - r\\ P(X=0,Y=0) &= 1 - P(X=0,Y=1) - P(X=1,Y=0) - P(X=1,Y=1) \\ &= 1 - (q - r) - (p - r) - r = 1 - p - q - r. \end{align*} $$ (On a side note, of course $r$ is bound to respect both $p-r\in[0,1]$, $q-r\in[0,1]$ and $1-p-q-r\in[0,1]$ beyond $r\in[0,1]$, which is to say $r\in[0,\min(p,q,1-p-q)]$.)
Notice that $r = P(X=1,Y=1)$ might be equal to the product $p\cdot q = P(X=1) P(Y=1)$, which would render $X$ and $Y$ independent, since $$ \begin{align*} P(X=0,Y=0) &= 1 - p - q - pq = (1-p)(1-q) = P(X=0)P(Y=0)\\ P(X=1,Y=0) &= p - pq = p(1-q) = P(X=1)P(Y=0)\\ P(X=0,Y=1) &= q - pq = (1-p)q = P(X=0)P(Y=1). \end{align*} $$
Yes, $r$ might be equal to $pq$, BUT it can be different, as long as it respects the boundaries above.
Well, from the above joint distribution, we would have $$ \begin{align*} E(X) &= 0\cdot P(X=0) + 1\cdot P(X=1) = P(X=1) = p \\ E(Y) &= 0\cdot P(Y=0) + 1\cdot P(Y=1) = P(Y=1) = q \\ E(XY) &= 0\cdot P(XY=0) + 1\cdot P(XY=1) \\ &= P(XY=1) = P(X=1,Y=1) = r\\ Cov(X,Y) &= E(XY) - E(X)E(Y) = r - pq \end{align*} $$
Now, notice then that $X$ and $Y$ are independent if and only if $Cov(X,Y)=0$. Indeed, if $X$ and $Y$ are independent, then $P(X=1,Y=1)=P(X=1)P(Y=1)$, which is to say $r=pq$. Therefore, $Cov(X,Y)=r-pq=0$; and, on the other hand, if $Cov(X,Y)=0$, then $r-pq=0$, which is to say $r=pq$. Therefore, $X$ and $Y$ are independent.
About the without loss of generality clause above, if $X$ and $Y$ were distributed otherwise, let's say, for $a<b$ and $c<d$, $$ \begin{align*} P(X=b)=p \\ P(Y=d)=q \\ P(X=b, Y=d)=r \end{align*} $$ then $X'$ and $Y'$ given by $$ X'=\frac{X-a}{b-a} \qquad \text{and} \qquad Y'=\frac{Y-c}{d-c} $$ would be distributed just as characterized above, since $$ X=a \Leftrightarrow X'=0, \quad X=b \Leftrightarrow X'=1, \quad Y=c \Leftrightarrow Y'=0 \quad \text{and} \quad Y=d \Leftrightarrow Y'=1. $$ So $X$ and $Y$ are independent if and only if $X'$ and $Y'$ are independent.
Also, we would have $$ \begin{align*} E(X') &= E\left(\frac{X-a}{b-a}\right) = \frac{E(X)-a}{b-a} \\ E(Y') &= E\left(\frac{Y-c}{d-c}\right) = \frac{E(Y)-c}{d-c} \\ E(X'Y') &= E\left(\frac{X-a}{b-a} \frac{Y-c}{d-c}\right) = \frac{E[(X-a)(Y-c)]}{(b-a)(d-c)} \\ &= \frac{E(XY-Xc-aY+ac)}{(b-a)(d-c)} = \frac{E(XY)-cE(X)-aE(Y)+ac}{(b-a)(d-c)} \\ Cov(X',Y') &= E(X'Y')-E(X')E(Y') \\ &= \frac{E(XY)-cE(X)-aE(Y)+ac}{(b-a)(d-c)} - \frac{E(X)-a}{b-a} \frac{E(Y)-c}{d-c} \\ &= \frac{[E(XY)-cE(X)-aE(Y)+ac] - [E(X)-a] [E(Y)-c]}{(b-a)(d-c)}\\ &= \frac{[E(XY)-cE(X)-aE(Y)+ac] - [E(X)E(Y)-cE(X)-aE(Y)+ac]}{(b-a)(d-c)}\\ &= \frac{E(XY)-E(X)E(Y)}{(b-a)(d-c)} = \frac{1}{(b-a)(d-c)} Cov(X,Y). \end{align*} $$ So $Cov(X,Y)=0$ if and only $Cov(X',Y')=0$.
=D
