55
$\begingroup$

I read from my textbook that $\text{cov}(X,Y)=0$ does not guarantee X and Y are independent. But if they are independent, their covariance must be 0. I could not think of any proper example yet; could someone provide one?

$\endgroup$
  • 11
    $\begingroup$ You might also enjoy a quick review of Anscombe's Quartet, which illustrates some of the many different ways in which a particular nonzero covariance can be realized by a bivariate dataset. $\endgroup$ – whuber Jul 11 '11 at 18:01
  • 7
    $\begingroup$ The thing to note is that the measure of covariance is a measure of linearity.. Calculating the covariance is answering the question 'Do the data form a straight line pattern?' If the data do follow a linear pattern, they are therefore dependent. BUT, this is only one way in which the data can be dependent. Its like asking 'Am I driving recklessly?' One question might be 'Are you travelling 25 mph over the speed limit?' But that isn't the only way to drive recklessly. Another question could be 'Are you drunk?' etc.. There is more than one way to drive recklessly. $\endgroup$ – Adam Feb 16 '12 at 4:13
  • $\begingroup$ The so- called measure of linearity gives a structure to the relationship. What is important that the relationship can be non-linear which is not uncommon. Generally, covariance is not zero, It is hypothetical.The covariance indicates the magnitude and not a ratio, $\endgroup$ – Subhash C. Davar Sep 24 '13 at 15:55
49
$\begingroup$

Easy example: Let $X$ be a random variable that is $-1$ or $+1$ with probability 0.5. Then let $Y$ be a random variable such that $Y=0$ if $X=-1$, and $Y$ is randomly $-1$ or $+1$ with probability 0.5 if $X=1$.

Clearly $X$ and $Y$ are highly dependent (since knowing $Y$ allows me to perfectly know $X$), but their covariance is zero: They both have zero mean, and

$$\eqalign{ \mathbb{E}[XY] &=&(-1) &\cdot &0 &\cdot &P(X=-1) \\ &+& 1 &\cdot &1 &\cdot &P(X=1,Y=1) \\ &+& 1 &\cdot &(-1)&\cdot &P(X=1,Y=-1) \\ &=&0. }$$

Or more generally, take any distribution $P(X)$ and any $P(Y|X)$ such that $P(Y=a|X) = P(Y=-a|X)$ for all $X$ (i.e., a joint distribution that is symmetric around the $x$ axis), and you will always have zero covariance. But you will have non-independence whenever $P(Y|X) \neq P(Y)$; i.e., the conditionals are not all equal to the marginal. Or ditto for symmetry around the $y$ axis.

$\endgroup$
33
$\begingroup$

Here is the example I always give to the students. Take a random variable $X$ with $EX=0$ and $EX^3=0$, e.g. normal random variable with zero mean. Take $Y=X^2$. It is clear that $X$ and $Y$ are related, but

$$cov(X,Y)=EXY-EX\cdot EY=EX^3=0.$$

$\endgroup$
  • $\begingroup$ I like that example too. As a particular case, a N(0,1) rv and a chi2(1) rv are uncorrelated. $\endgroup$ – ocram Jul 15 '11 at 8:21
  • 3
    $\begingroup$ +1 but as a minor nitpick, you do need to assume that $E[X^3] = 0$ separately (it does not follow from the assumption of symmetry of the distribution or from $E[X] = 0$), so that we don't have issues such as $E[X^3]$ working out to be of the form $\infty - \infty$. And I am queasy about @ocram's assertion that "a N(0,1) rv and a chi2(1) rv are uncorrelated." (emphasis added) Yes, $X \sim N(0,1)$ and $X^2 \sim \chi^2(1)$ are uncorrelated, but not any $N(0,1)$ and $\chi^2(1)$ random variables. $\endgroup$ – Dilip Sarwate Feb 16 '12 at 3:09
  • $\begingroup$ @DilipSarwate, thanks, I've edited my answer accordingly. When I wrote it I though about normal variables, for them zero third moment follows from zero mean. $\endgroup$ – mpiktas Feb 16 '12 at 8:25
19
$\begingroup$

The image below (source Wikipedia) has a number of examples on the third row, in particular the first and the fourth example have a strong dependent relationship, but 0 correlation (and 0 covariance).

enter image description here

$\endgroup$
15
$\begingroup$

Some other examples, consider datapoints that form a circle or ellipse, the covariance is 0, but knowing x you narrow y to 2 values. Or data in a square or rectangle. Also data that forms an X or a V or a ^ or < or > will all give covariance 0, but are not independent. If y = sin(x) (or cos) and x covers an integer multiple of periods then cov will equal 0, but knowing x you know y or at least |y| in the ellipse, x, <, and > cases.

$\endgroup$
  • 1
    $\begingroup$ That if should be "if x covers an integer multiple of periods beginning at a peak or trough", or more generally: "If x covers an interval on which y is symmetric" $\endgroup$ – naught101 Feb 16 '12 at 0:47
  • $\begingroup$ could you explain why the covariance is zero for a circle? $\endgroup$ – user1993 Nov 6 at 20:32
  • 1
    $\begingroup$ @user1993, Look at the formula for covariance (or correlation). Then think about the circle/ellipse. Subtracting the means gives a circle centered on (0,0), so for every point on the circle you can reflect the point around the x-axis, the y-axis, and both axes to find a total of 4 points that will all contribute the exact same absolute value to the covariance, but 2 will be positive and 2 will be negative giving a sum of 0. Do this for all of the points on a circle and you will be adding together a bunch of 0's giving a total covariance of 0. $\endgroup$ – Greg Snow Nov 6 at 20:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.