4
$\begingroup$

I have two variables whose correlation coefficient $\rho=1$. I know that if two random variables are a linear combination of themselves, it implies $|\rho |=1$. Is the converse also true? How to prove that?

$\endgroup$
3
$\begingroup$

whuber's much more detailed answer appeared while I was composing this answer of mine (which essentially uses the same argument).

Let $X$ and $Y$ denote two random variables with finite variances $\sigma_X^2$ and $\sigma_Y^2$ respectively and correlation coefficient $\rho = \pm 1$. Then, \begin{align}\operatorname{var}(Y-aX) &= \sigma_Y^2+ a^2\sigma_X^2 - 2a\cdot\operatorname{cov}(Y,X) &\text{standard result}\\ &= \sigma_Y^2+ a^2\sigma_X^2 - 2a\rho\sigma_X\sigma_Y &\text{substitute for}~\operatorname{cov}(Y,X)\\ &= \sigma_Y^2+ a^2\sigma_X^2 \mp 2a\sigma_X\sigma_Y & \text{since}~ \rho = \pm 1\\ &= (\sigma_Y\mp a\sigma_X)^2\\ &= (\sigma_Y - a\rho\sigma_X)^2 & \text{keep remembering that}~ \rho = \pm 1\\ &= 0 &\text{if we choose}~ a = \rho\frac{\sigma_Y}{\sigma_X}. \end{align} Thus, if $\rho = \pm 1$, then $Y-\rho\frac{\sigma_Y}{\sigma_X}X$ is a random variable whose variance is $0$, and so $Y-\rho\frac{\sigma_Y}{\sigma_X}X$ is a constant (almost surely). In other words, $Y = \alpha X + \beta$ (almost surely) and thus $X$ and $Y$ are linearly related (almost surely).

$\endgroup$
3
$\begingroup$

Let the pair of (random) variables be $(X_1,X_2)$. Since their correlation coefficient exists, each has a finite variance $\sigma^2_i$ and a finite mean $\mu_i$. The standardized variables are $Z_i = (X_i - \mu_i)/\sigma_i$. In particular their second moments are unity:

$$\mathbb{E}(Z_i^2) = 1.$$

By definition, the correlation

$$\rho = \rho(X_1,X_2) = \mathbb{E}(Z_1Z_2)$$

is the expected product of the standardized variables. Since for any real numbers $x$, $y$, and $\rho$ it is the case that

$$2\rho x y = x^2 + \rho^2 y - (x-\rho y)^2,$$

and expectation is linear, we may compute

$$\eqalign{ 0 &= 2-2\rho^2 = 2 - 2\rho\, \mathbb{E}(Z_1Z_2) = 2- \mathbb{E}(2\rho Z_1Z_2) \\ &= 2-\mathbb{E}\left(Z_1^2 + Z_2^2- (Z_1 - \rho Z_2)^2\right) \\ &= 2-\left(1 + 1 - \mathbb{E}((Z_1 - \rho Z_2)^2)\right) \\ &= \mathbb{E}((Z_1 - \rho Z_2)^2). }$$

Now--for the first time--we need to invoke a basic result about random variables: because $(Z_1 - \rho Z_2)^2$ has zero expectation, $Z_1 - \rho Z_2 = 0$ almost surely. (A direct application of Chebyshev's Inequality will prove this.)

Unraveling the algebra to re-express the $Z_i$ in terms of the $X_i$, we find

$$\sigma_2 X_1 - \rho \sigma_1 X_2 + (\rho \sigma_1 \mu_2 - \sigma_2 \mu_1) = 0$$

almost surely. This is a linear relation between $X_1$ and $X_2$. It is not quite the result requested in the question, because it involves an additive constant $\rho \sigma_1 \mu_2 - \sigma_2 \mu_1$, whereas linear combinations do not. The constant is unavoidable: for instance, $X_1$ and $X_1+1$ have unit correlation but $X_1+1$ is not a linear combination of $X_1$.

What we have proven to be true is

When $|\rho(X_1,X_2)|=1$, $X_1$ and $X_2$ are linearly related almost surely. The coefficients in the relation are universal algebraic combinations of the first two moments of the variables.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.