6
$\begingroup$

I get that a zero covariance doesn´t imply independence, but everybody says that if there is dependence and the covariance is zero then it is a non linear dependence.

People base their interpretation of Pearson's R in that fact (the closer you are to zero the less linear the relationship is).

Is there a formal proof to that?

I tried to do it by myself but i couldn't. The proposition i think encapsulates the idea is the following:

If $cov(X,Y)\ne0$ then there exists a Z such that $cov(X,Z)=0$ and $E[Y|X]=bX+E[Z|X]$

$\endgroup$
  • 6
    $\begingroup$ It seems to me that this follows directly from the definition of Pearson's correlation coefficient: $$\rho_{X,Y}=\frac{\mathsf{Cov}(X,Y)}{\sigma_X\sigma_Y}$$ If the covariance is zero then the correlation is zero and so they're not linearly related. $\endgroup$ – Remy Oct 19 at 5:41
  • 1
    $\begingroup$ Interesting, that means i should turn my attention into what inspired that exact definition. It couldn´t come out of nowhere. $\endgroup$ – Gilbert Ibanez Oct 19 at 18:07
  • $\begingroup$ I think the proposition you put at the end isn't quite right. You want if $cov(X, Y) = 0$ then no linear relationship. But you haveif $cov(X, y) \neq 0$ then linear relationship $\endgroup$ – roundsquare Oct 31 at 20:53
2
$\begingroup$

Here is a proof of the mathematical statement at the end of your question: we can find a $Z$ which is uncorrelated to $X$ and satisfies $$ \mathbb{E}(Y|X) = b X + \mathbb{E}(Z|X) $$ by assuming $Z = Y - bX$, and then choosing the $b$ which makes $\mathrm{Cov}(X, Z) = 0$ true. For this $b$ we have $$ 0 = \mathrm{Cov}(X, Z) = \mathrm{Cov}(X, Y - bX) = \mathrm{Cov}(X, Y) - b \mathrm{Var}(X), $$ and thus $$ b = \frac{\mathrm{Cov}(X, Y)}{\mathrm{Var}(X)}. $$ (Note that the same $b$ is found as the slope of the linear regression line.) We have $b = 0$, if and only if $\mathrm{Cov}(X,Y) = 0$.

$\endgroup$
  • 1
    $\begingroup$ wow, it looks "to easy to be true" but i think it is. You also showed that if we choose that specific $Z$ then it is necesarry for $b$ to be $cov(X,Y)/var(X)$. Maybe if someone else constructs a different $Z$, a different requirement for $b$ arises. But i can live with your approach since it shows the best linear approximation, according to linear regression criteria. $\endgroup$ – Gilbert Ibanez Oct 20 at 18:28
8
$\begingroup$

If there is a linear relationship between two RVs, i.e. $Y=aX+b$, where $a\neq 0$, then the covariance is $$\operatorname{cov}(X,Y)=a\operatorname{cov}(X,X)=a\operatorname{var}(X)\neq0$$ So, if there a linear relation, covariance is not zero. If the covariance is zero, the linear relation can't exist because we'll contradict.

$\endgroup$
  • 2
    $\begingroup$ This seems as a very specific situation. What if that b was another random variable. Since covariance can be negative it could be the case that cov(B,X)=-aV[X] which makes cov(X,Y)=0 $\endgroup$ – Gilbert Ibanez Oct 19 at 7:56
  • $\begingroup$ If $b$ is random, we can't say anything; Let $Y=X+b$, then what happens when $b=-X$, which is similar to your case actually? $\endgroup$ – gunes Oct 19 at 8:14
  • $\begingroup$ I believe that if we say that if $b$ is random and uncorrelated to $X$ then what you say holds. Thats why I believe we need to find at least one $b$ with that property. Thats why in my question i resume this issue as a proof of existence(with the only difference that i used Z instead of B) $\endgroup$ – Gilbert Ibanez Oct 19 at 18:05
  • 1
    $\begingroup$ If $b$ is random, in general we can't say something. And, yes, if $b$ is random and uncorrelated to $X$, and $X$ is lin. related to $Y$, covariance is non-zero. However, there are a lot of situations. If, let's say $X$ is zero mean, and $b=X^2$, covariance of $Y$ and $X$ will be nonzero, where $cov(b,X)=0$. The relation is actually $Y=X+X^2$, we have a linear term, but is this a linear relationship in the end? $\endgroup$ – gunes Oct 20 at 6:36
  • $\begingroup$ Excellent point, $cov(x,y) \ne 0$ and $Y=X+X^2$ is not a perfect linear relationship. However, i feel that there is some linearity in it, since it can be (badly) approximated by a straight line. I got that feeling because the covariance is not zero, but i can´t formally explain that fact. $\endgroup$ – Gilbert Ibanez Oct 20 at 18:08
3
$\begingroup$

On a distribution level it should be straightforward to show that a linear correlation implies a non-zero covariance (the other way to prove what you wanted).

But as a word of warning, this may not hold for a sample. If you have a small data set generated with a linear correlation, but by chance a large outlier you can compute a negative correlation or no correlation on the sample.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.