Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Can somebody illustrate, as Greg does, but in more detail, how random variables can be dependent but have zero covariance? Greg, a poster here, gives an example using a circle here.

Can somebody explain this process in more detail using a sequence of steps that illustrate the process at several stages? Thanks. Also, if you know of an example from psychology, please illustrate with this concept with a related example. Please be very precise and sequential in your explanation, and also state what some of the consequences might be. Thank you.

share|improve this question
    
A link to the article you refer to would help. –  gui11aume Jun 9 '12 at 20:30
1  
I'm pretty sure the topic is here: stats.stackexchange.com/questions/12842/… and the answer by Greg (Snow) is here: stats.stackexchange.com/a/12898/2073 –  Andy McKenzie Jun 9 '12 at 20:30
    
I initially thought closing this question was correct but I think perhaps it asks a subtlely different question from here stats.stackexchange.com/questions/12842/… since that thread just asked for mathematical examples, while this question appears to seek an understanding of exactly why the two are not equivalent –  Macro Jun 9 '12 at 21:40
add comment

2 Answers

The basic idea here is that covariance only measures one particular type of dependence, therefore the two are not equivalent. Specifically,

  • Covariance is a measure how linearly related two variables are. If two variables are non-linearly related, this will not be reflected in the covariance. A more detailed description can be found here.

  • Dependence between random variables refers to any type of relationship between the two that causes them to act differently "together" than they do "by themselves". Specifically, dependence between random variables subsumes any relationship between the two that causes their joint distribution to not be the product of their marginal distributions. This includes linear relationships as well as many many others.

  • If two variables are non-linearly related, then they can potentially have 0 covariance but are still dependent - many examples are given here and this plot below from wikipedia gives some graphical examples in the bottom row:

    enter image description here

  • One example where zero covariance and independence between random variables are equivalent conditions is when the variables are jointly normally distributed (that is, the two variables follow a bivariate normal distribution, which is not equivalent to the two variables being individually normally distributed). Another special case is that pairs of bernoulli variables are uncorrelated if and only if they are independent (thanks @cardinal). But, in general the two cannot be taken to be equivalent.

Therefore, one cannot, in general, conclude that two variables are independent just because they appear uncorrelated (e.g. didn't fail to reject the null hypothesis of no correlation). One is well advised to plot data to infer whether the two are related, not just stopping at a test of correlation. For example, (thanks @gung), if one were to run a linear regression (i.e. testing for non-zero correlation) and found a non-sigificant result, one may be tempted to conclude that the variables are not related, but you've only investigated a linear relationship.

I don't know much about Psychology but it makes sense that there could be non-linear relationships between variables there. As a toy example, it seems possible that cognitive ability is non-linearly related to age - very young and very old people are not as sharp as a 30 year old. If one were to plot some measure of cognitive ablity vs. age one may expect to see that cognitive ability is highest at a moderate age and decays around that, which would be a non-linear pattern.

share|improve this answer
    
Just a side (pedantic?!‌​) note, but Bernoulli random variables are independent if and only if they are uncorrelated. :) –  cardinal Jun 10 '12 at 14:42
    
@cardinal, don't worry that's just me abandoning rationality again, kind of like when you said that a multivariate normal with a singular covariance matrix was "commonly used and statistically relevant". –  Macro Jun 10 '12 at 14:43
    
The next time I'm in Ann Arbor, I'll buy you a coffee to try to offset that joke. :) Feel free to question my rationality in the meantime. :) –  cardinal Jun 10 '12 at 14:49
    
Ah, but that last citation is true. ;-) It shows up in some surprisingly common places. :) (Though that's getting a little off-topic here.) –  cardinal Jun 10 '12 at 14:49
    
(+1) I have been kind of on the fence about whether this question should be closed as a duplicate or not. But, I think good answers can make very similar questions worth keeping. Having everything cross-linked helps. –  cardinal Jun 10 '12 at 15:45
add comment

A standard way of teaching/visualizing a correlation or covariance is to plot the data, draw lines at the mean of 'x' and 'y', then draw rectangles from the point of the 2 means to the individual datapoints, like this:

enter image description here

The rectangles (points) in the top right and bottom left quadrants (red in the example) contribute positive values to the correlation/covariance, while the rectangles (points) in the top left and bottom right quadrants (blue in the example) contribute negative values to the correlation/covariance. If the total area of the red rectangles equals the total area of the blue rectangles then the positives and negatives cancel out and you get a zero covariance. If there is more area in the red then the covariance will be positive and if there is more area in the blue then the covariance will be negative.

Now lets look at an example from the previous discussion:

enter image description here

The individual points follow a parabola, so they are dependent, if you know 'x' then you know 'y' exactly, but you can also see that for every red rectangle there is a matching blue rectangle, so the final covariance will be 0.

share|improve this answer
    
(+1) is there an R package that makes these plots (I recall whuber displaying a plot like this once) or did you do this from scratch? –  Macro Jun 11 '12 at 17:34
    
@Macro, good question, though I think whuber's were done in Mathematica. It is straightforward to do this "by hand" in R using polygon or rect and a device that supports alpha transparency. –  cardinal Jun 11 '12 at 17:43
    
I wrote a function to do this plot and will probably add it to the TeachingDemos package sometime soon. My first thought was to shorten the phrase "correlation rectangles" to "correct" as the name of the function, then after a bit realized that name may be easily misunderstood as doing something quite different. So I need to come up with a better name, add a couple options and upload it to R-Forge. –  Greg Snow Jun 11 '12 at 21:14
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.