2
$\begingroup$

I know there are already several posts about the relationship between covariance and independence in this forum but I couldn’t find the answer to the issue bugging me on those posts.

Let me firstly say I know that: (1) independence between RVs means covariance = 0, but covariance = 0 does not necessarily mean independency between RVs. (2) Pearson’s correlation coefficient (corrcoef) detects linear dependencies between RVs, so non-linear dependencies might well yield covariance = 0 (a simple example is the parabola one, when RVs are X and Y = X^2).

Now, let X and Y be RVs, where X = A + B and Y = A - B; Here, A and B are also RVs, that could be either dependent and independent, and with zero-mean, variance = 1.

What I want to know is whether X and Y are independent (or dependent).

At first glance I am inclined to say X and Y are dependent because both are functions of the same A and B. I would also say that their relationship seems linear because Y = X - 2B. What is making me confused is that COV(X,Y) = 0, so, corrcoef = 0, as can be seen below.

COV(X,Y) = E[(X - E[X])(Y - E[Y])] = E[XY] = E[(A+B)(A-B)] = E[A^2 - B^2] = E[A^2] - E[B^2] = 1 - 1 = 0 or COV(A+B,A-B) = COV(A,A) + COV(A,-B) + COV(B,A) + COV(B,-B) = VAR(A) - COV(A,B) + COV(A,B) - VAR(B) = 1 - COV(A,B) + COV(A,B) - 1 = 0

Wouldn’t COV(X,Y) = 0 mean that X and Y are either independent or dependent though following a non-linear relationship? But they don’t seem independent and they don’t seem to have a non-linear relationship. They seem to have a linear relationship, but in such case covariance should be different from 0.

What is the issue here? I would be glad if someone could shed some light on that.

Improve this question
$\endgroup$
2
  • $\begingroup$ If you want a very silly example you can take $A = 0$ and $B=0$, i.e. $A,B$ are "random" variables which always produce the constant value of zero. Then $X=A+B=0$ and $Y = A-B = 0$ also. You can then check that $X$ and $Y$ are independent, indeed, $P(X = x, Y = y) = 0$ unless $x,y = 0$ in which case $P(X=0,Y=0) = 1$, i.e. it is equal to the product $P(X=0)P(Y=0)$. $\endgroup$
    – Nicolas Bourbaki
    Dec 24, 2022 at 22:29
  • $\begingroup$ @NicolasBourbaki Thank you for your reply. Indeed you pointed out a case in which X and Y are independent. In fact, elaborating a bit on your example, I think that any constant RVs A,B, would result in X,Y being independent... For instance, if A, B are "RVs" that always produce A = B = 2, then 𝑃(𝑋=𝑥,𝑌=𝑦)=0 unless 𝑥=4,𝑦=0 in which case 𝑃(𝑋=4,𝑌=0)=1, i.e. it is equal to the product 𝑃(𝑋=4)𝑃(𝑌=0)= 1x1 = 1. So, how should I interprete your answer, are you saying that X and Y are, in general, dependent and providing one exceptional case (constant RVs) in which they become independent? $\endgroup$
    – Robb
    Dec 26, 2022 at 7:12

1 Answer 1

Reset to default
1
$\begingroup$

Wouldn’t COV(X,Y) = 0 mean that X and Y are either independent or dependent though following a non-linear relationship?

You answered this yourself

but covariance = 0 does not necessarily mean independency between RVs

An example of a case where X,Y are independent is when $A,B$ are i.i.d normal distributed variables.

An example of a case where X,Y are dependent is when $A,B$ have a joint distribution that is spherically symmetric but which is not a normal distribution.

Below are example plots for the two situations. Covariance is zero for $X$ and $Y$ (and also for $A$ and $B$), and in the first case there is independence but in the second case there is no independence.

example

In these examples I chose spherical symmetrical joint distributions since for that case the transformations $A+B$ and $A-B$ are just the same as a rotation (and a scaling). The $X$ and $Y$ will have the same dependence as the $A$ and $B$.

Improve this answer
$\endgroup$
12
  • $\begingroup$ Thanks a lot for the answer and illustrations. So, you are saying that X and Y could be either dependent or independent, as a result of A and B. Sorry, could you clarify why X and Y would be independent if A and B are i.i.d.? Y = X - 2B, so they look dependent to me regardless of A and B relationship. And, the the real problem I am having is that Y = X - 2B seems a "linear" relationship, which should have been detected by the Pearson corrcoef, which would result in COV(X,Y) not being 0. Is Y = X - 2B a non-linear dependency? $\endgroup$
    – Robb
    Dec 25, 2022 at 18:06
  • $\begingroup$ Or more precisely, are you saying that Y = X - 2B does not mean dependency when A,B are i.i.d; and Y = X - 2B means non-linear dependency, for other types of A,B? Sorry, but I don't quite understand. $\endgroup$
    – Robb
    Dec 25, 2022 at 18:17
  • $\begingroup$ @Robb Say $Y$ and $X$ are independent, and we define $2B = X - Y$ then we can write $Y = X - 2B$, but just that equation doesn't make $Y$ and $X$ dependent. The expression can be rewritten as $$Y = X - 2B = X - (X-Y) = Y$$ Because $B$ and $X$ are dependent the seamingly dependence of $Y$ on $X$ cancels out. You have an expression with an $X$ on the one side but there is another $-X$ hidden in there in the $-2B$. $\endgroup$
    – Sextus Empiricus
    Dec 25, 2022 at 19:31
  • $\begingroup$ Thank you. I got that Y= X - 2B (resultant from X= A+B; Y= A-B) does not necessarily mean X and Y are dependent by the reason you explained in your comment. But, according to your answer, in case A,B are non i.i.d, then X,Y are dependent. And in such case, the dependency should be given by Y = X-2B. Then, the question becomes: is Y = X-2B a non-linear dependency? I looks very linear to me, but on the other hand COV(X,Y) = 0, as shown above in my post, means X and Y are not linearly dependent. Is there something "under the hood" that makes Y = X - 2B non-linear and that I am missing? $\endgroup$
    – Robb
    Dec 26, 2022 at 8:13
  • $\begingroup$ @Robb "But, according to your answer, in case A,B are non i.i.d, then X,Y are dependent." That's only in my example, I am not saying that this must generally be the case. Example let U,V be two iid dice rolls. Let A = 0.5(U+V) and B = 0.5(U-V). Then A and B are not independent and neither identical, but you do have that A+B and A-B are iid. $\endgroup$
    – Sextus Empiricus
    Dec 26, 2022 at 8:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.