13
$\begingroup$

In a problem I have found that $$Cov(X,Y)=Var(Y),$$ where $X$ and $Y$ are random variables. What can I conclude on the linear dependence between $X$ and $Y$? Thank you!

$\endgroup$

5 Answers 5

17
$\begingroup$

If we let $\rho_{X,Y}$ denote the correlation between the variables and let $s_X$ and $s_Y$ denote their respective standard deviations then we have:

$$\rho_{X,Y} s_X s_Y = \mathbb{Cov}(X,Y) = \mathbb{V}(Y) = s_Y^2,$$

which then reduces to:

$$\rho_{X,Y} s_X = s_Y.$$

Any pair of variables $X$ and $Y$ which satisfy this moment equation will satisfy the equation of interest to you. Consequently, any of the following situations will yield the result of interest to you:

$$\begin{matrix} s_X = s_Y = 0 & & \implies & & \mathbb{Cov}(X,Y) = \mathbb{V}(Y), \\[6pt] \rho_{X,Y} = s_Y = 0 & & \implies & & \mathbb{Cov}(X,Y) = \mathbb{V}(Y), \\[6pt] \rho_{X,Y} = s_Y/s_X & & \implies & & \mathbb{Cov}(X,Y) = \mathbb{V}(Y). \\[6pt] \end{matrix}$$

(As others have pointed out, the antecedent condition in the last statement requires $s_Y \leqslant s_X$ and $\rho_{X,Y} \geqslant 0$, so the variables cannot be negatively correlated and the standard deviation of $Y$ cannot be larger than the standard deviation of $X$.)

$\endgroup$
8
  • 1
    $\begingroup$ The first two cases can be considered pathological, so the most interesting Is the third. It seems that you have to add the restriction $Sy <= Sx$, otherwise $\rho$ greater than $1$ seems possible. $\endgroup$
    – markowitz
    Dec 28, 2023 at 8:54
  • 1
    $\begingroup$ @markowitz That restriction is unnecessary because it's vacuous: even when $s_Y\gt s_X,$ the assumption $\rho_{X,Y} = s_Y/s_X$ still implies the conclusion (as well as any meaningful conclusion at all, whether true or false, according to the laws of logic). $\endgroup$
    – whuber
    Dec 28, 2023 at 18:44
  • $\begingroup$ @whuber if $Sx < Sy$ the assumpion $\rho = Sy / Sx$ cannot be true. Moreover It seems me that in this case even $cov(x,y)=v(y)$ cannot holds. $\endgroup$
    – markowitz
    Dec 28, 2023 at 19:50
  • 2
    $\begingroup$ @Markowitz Any conditional statement in which the antecedent is false is an automatically true statement: that's one of the axioms of logic. Thus, when an assumption cannot be true, any conditional statement (in which that assumption is the antecedent) is automatically true, regardless of the truth value of the consequent. $\endgroup$
    – whuber
    Dec 28, 2023 at 20:06
  • 2
    $\begingroup$ @markowitz: What whuber is (correctly) saying is that although you are correct that $s_Y > s_X$ makes $\rho_{X,Y} = s_Y/s_X$ impossible, this isn't a restriction you need to state in the implication statement (just as you also don't need to state the requirement $\rho_{X,Y} \geqslant 0$). If the antecedent condition $\rho_{X,Y} = s_Y/s_X$ is false (for whatever reason) then the implication statement says nothing. $\endgroup$
    – Ben
    Dec 28, 2023 at 20:20
15
$\begingroup$

Concerning the question of linear dependence, you can conclude only that the variables are positively related.

From the usual formulas, with $s_*$ representing standard deviations and $\rho$ the (Pearson) correlation coefficient, your relation $$\operatorname{Cov}(X,Y)=\operatorname{Var}(Y)\tag{*}$$ is equivalent to

$$s_X s_Y \rho = s_Y^2.$$

(When $s_Y\ne 0,$ obviously this is equivalent to $s_X \rho = s_Y.$ Since $|\rho|\le 1$ this has many evident implications about the relative magnitudes of the standard deviations, as discussed in comments -- but those are tangential issues with no bearing on the linear dependence between $X$ and $Y.$)

Because the $s_*$ are positive (for otherwise $\rho$ would be undefined), it is immediate that

$$\rho \gt 0.$$

To show that we cannot deduce any other restrictions, observe that for any positive value of $\rho$ we can find variables $X$ and $Y$ for which $(*)$ holds. For instance, let $U$ and $V$ be any uncorrelated, non-constant variables (with finite variances $s_U^2$ and $s_V^2$) and set

$$\cases{X = \frac{1}{ \rho}\left(\frac{\rho}{s_U} U + \frac{\sqrt{1 - \rho^2}}{s_V}V\right),\\ Y = \frac{1}{s_U}U.}$$

Compute that

$$\operatorname{Var}(X) = \frac{1}{\rho^2},\quad \operatorname{Var}(Y) = 1,\quad \rho(X,Y) = \rho,$$

whence

$$s_Xs_Y\rho = \left(\frac{1}{\rho}\right)(1)(\rho) = 1 = s_Y^2.$$

$\endgroup$
4
  • $\begingroup$ Your conclusion sound like: $cov(x,y)=var(y)$ for any couple of positively correlated r.vs. but this is not true. Would you add something for avoid this missunderstanding? $\endgroup$
    – markowitz
    Dec 27, 2023 at 22:07
  • $\begingroup$ Suppose $Y=U$ and $X=U+V$. Then $\rho = \frac{s_U}{s_{U+V}}$. Since $U \perp V$, $s_{U+V}=s_U+s_V$. So $\rho=\frac{s_U}{s_U+s_V}$ and $s_V = \frac{s_U}{\rho}-s_U$. So given any $Y$ and $\rho$, we can simply pick any $V$ that's perpendicular to $Y$ with $s_V= \frac{s_Y}{\rho}-s_Y$, and the correlation between $Y$ and $X$ will be equal to $\rho$. If $\rho = 1$, then $s_V$ has to be zero. As $\rho$ goes to zero, $s_V$ goes to infinity. $\endgroup$ Dec 28, 2023 at 6:37
  • $\begingroup$ @markowitz All the implications are one-way. $\endgroup$
    – whuber
    Dec 28, 2023 at 14:53
  • $\begingroup$ @Acccumulation If you are using my notation, then your statement goes awry near the beginning where you assert the standard deviation is additive. $\endgroup$
    – whuber
    Dec 28, 2023 at 14:54
13
$\begingroup$

If $Var(Y) = 0$, then the statement $Cov(X,Y)=Var(Y)$ holds for all variables $X$, so let's consider only the case where $Var(Y)>0$.

Considering $X$ and $Y$ as vectors, we have that $X \cdot Y = Y \cdot Y$ where $\cdot$ is the dot product. We can decompose $X$ into $X_{\perp}+X_{\parallel}$ where $X_{\perp} \cdot Y = 0$ and $X_{\parallel} = cY$ for some scalar $c$. That is, $X_{\perp}$ has zero correlation with $Y$, and $X_{\parallel}$ has correlation $1$ with $Y$.

We then have $$(X_{\perp}+X_{\parallel}) \cdot Y = Y \cdot Y$$ $$(X_{\perp} \cdot Y)+(X_{\parallel} \cdot Y) = Y \cdot Y$$ $$X_{\parallel} \cdot Y = Y \cdot Y$$ $$cY \cdot Y = Y \cdot Y$$ $$c = 1$$

Thus, we can conclude that $X = Y+Z$ where $Y$ and $Z$ have zero correlation.

Another way of saying this is that $X$ projected onto $Y$ gives $Y$.

$\endgroup$
1
  • $\begingroup$ Many thanks! It is very useful. Do you know a reference for this result? $\endgroup$
    – Jo R
    Dec 28, 2023 at 13:14
7
$\begingroup$

In his masterly answer (now revised), whuber shows that with regard to the question of linear dependence between $X$ and $Y$, the only conclusion that can be drawn from the hypothesis that $\operatorname{cov}(X,Y)= \operatorname{var}(Y)$ is that $\rho > 0$. There are also other conclusions that one can reach, such as $\operatorname{var}(X)\geq \operatorname{var}(Y)$ (as @Ben has already done) by noting that $$\require{cancel}0 \leq\operatorname{var}(X-Y)= \operatorname{var}(X)+ \operatorname{var}(Y) -2\cancelto{\operatorname{var}(Y)}{\operatorname{cov}(X,Y)} = \operatorname{var}(X)- \operatorname{var}(Y)$$ implies that $\operatorname{var}(X)\geq \operatorname{var}(Y)$ and, as noted in markowitz's comment, that $\rho = \displaystyle\frac{\operatorname{var}(Y)}{\operatorname{var}(X)}$ which confirms that not only is $\rho > 0$ but also that $\rho \leq 1$, as it should be.

$\endgroup$
6
  • $\begingroup$ The condition $\rho = Sy/Sx$ Is another relevant deduction. $\endgroup$
    – markowitz
    Dec 29, 2023 at 8:43
  • $\begingroup$ You misinterpret my answer. In the context of the question, which asks "conclude [about] the linear dependence," the "only" refers to the linear dependence. The relationship between the variances is not an aspect of the question. $\endgroup$
    – whuber
    Dec 29, 2023 at 14:31
  • 1
    $\begingroup$ @whuber I apologize for misunderstanding your answer and will soon edit my answer appropriately $\endgroup$ Dec 29, 2023 at 16:04
  • 1
    $\begingroup$ And I, in turn, will soon clarify my answer! $\endgroup$
    – whuber
    Dec 29, 2023 at 16:05
  • $\begingroup$ +1. The logic of your argument would be clearer by starting the sequence of equations with $0\le \operatorname{Var}(X-Y) = \cdots.$ $\endgroup$
    – whuber
    Dec 29, 2023 at 17:01
-3
$\begingroup$

If you have a r.v. $Y$ with finite variance and another r.v. $X = a + Y$ you have that your condition hold. Indeed: $Cov[X,Y]=Cov[a+Y,Y]=Cov[Y,Y]=V[Y]$

This seems me the most intuitive case.

However more in general, for any couple of r.vs. $(X,Y)$ with positive but finite variance, we have $Cov[X,Y]=corr[X,Y]sd[X]sd[Y]$

now, if we apply the restriction $Cov[X,Y]=V[Y]$

we can deduce that $Corr[X,Y]=sd[Y]/sd[X]$ holds, then $0<corr[X,Y]\leqslant 1$

moreover we can deduce even that $sd[Y] \leqslant sd[X]$ must holds.

Note that the case considered initially, the most intuitive in my opinion, is comply with the general rule. Indeed represent the case $sd[Y]=sd[X]$ and $corr[X,Y]=1$

Finally consider that the more general deduction do not mean that "for any couple of positively correlated r.vs. $Cov[X,Y]=V[Y]$ holds. It mean that, if $Cov[X,Y]=V[Y]$ holds, the correlation is positive but even the above condition must be respected ... a much stronger restriction.

@downvoters to leave your suggestions/critics in comments and wait a couple of days seems me a more correct prassi. I cannot spend all day here.

$\endgroup$
6
  • 1
    $\begingroup$ Right! I edited $\endgroup$
    – markowitz
    Dec 27, 2023 at 21:04
  • $\begingroup$ I disagree with this conclusion and have explained why in an answer to this thread. $\endgroup$
    – whuber
    Dec 27, 2023 at 21:14
  • 2
    $\begingroup$ This answer anyway requires a lot more explanation, regardless of if its right or wrong. $\endgroup$ Dec 28, 2023 at 6:48
  • 2
    $\begingroup$ This is not true. Counterexample: let $Z$ be any non-constant random variable independent of $Y$, and let $X = Y+Z$. $\endgroup$ Dec 28, 2023 at 7:53
  • $\begingroup$ The case already given Is surely adeguate. Let me time to think about other case. $\endgroup$
    – markowitz
    Dec 28, 2023 at 8:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.