9
$\begingroup$

Is it possible to have a set of $K$ variables that are uncorrelated but linearly dependent?

i.e. $cor(x_i, x_j)=0$ and $ \sum_{i=1}^K a_ix_i=0$

If yes can you write an example ?

EDIT: From the answers it follows that it is not possible.

Would it at least be possible that $\mathbb{P}(|\hat \rho_{x_i, x_j}-\hat \rho_{x_i, v}|<\epsilon)$ where $\hat\rho$ is the estimated correlation coefficient estimated from $n$ samples of the variables and $v$ is a variable that is uncorrelated with $x_i$.

I am thinking something like $x_K=\dfrac{1}{K} \sum_{i=1}^{K-1} x_i$ $K>>0$

$\endgroup$
11
$\begingroup$

As @RUser4512's answer shows, uncorrelated random variables cannot be linearly dependent. But, nearly uncorrelated random variables can be linearly dependent, and one example of these is something dear to the statistician's heart.

Suppose that $\{X_i\}_{i=1}^K$ is a set of $K$ uncorrelated unit-variance random variables with common mean $\mu$. Define $Y_i = X_i - \bar{X}$ where $\bar{X} = \frac 1K \sum_{i=1}^K X_i$. Then, the $Y_i$ are zero-mean random variables such that $\sum_{i=1}^K Y_i = 0$, that is, they are linearly dependent. Now, $$Y_i = \frac{K-1}{K} X_i - \frac 1K\sum_{j \neq i}X_j$$ so that $$\operatorname{var}(Y_i) = \left(\frac{K-1}{K}\right)^2+\frac{K-1}{K^2} = \frac{K-1}{K}$$ while $$\operatorname{cov}(Y_i,Y_j) = -2\left(\frac{K-1}{K}\right)\frac 1K + \frac{K-2}{K^2}= \frac{-1}{K}$$ showing that the $Y_i$ are nearly uncorrelated random variables with correlation coefficient $\displaystyle -\frac{1}{K-1}$.

See also this earlier answer of mine.

$\endgroup$
  • 1
    $\begingroup$ This is a really nice example! $\endgroup$ – RUser4512 Oct 1 '15 at 14:52
9
$\begingroup$

No.

Suppose one of the $a_i$ is non zero. Without loss of generality, let's assume $a_1=1$.

For $K=2$, this implies $x_1=-a_2x_2$ and $cor(x_1,x_2)=-1$. But this correlation is zero. $a_1$ must be zero as well, contradicting the existence of a linear relationship.

For any $K$, $x_1=-\sum_{i>1}a_ix_i$ and $cor(x_1,x_k)=-1$. But, by you hypothesis, $cor(x_1,x_k)=0$. The $a_i$'s are zero (for $i>1$) and so must be $a_1$.

$\endgroup$
  • $\begingroup$ In the case of gaussian vectors, you even have a one-line proof (that I prefer to keep as a comment). Correlation equals 0 implies independence. $\sum_i a_ix_i =0$ implies $\sum_i a_i^2 =0$ and you are done. $\endgroup$ – RUser4512 Oct 1 '15 at 12:36
  • $\begingroup$ Very good answer. It would be nice if you can answer also to the edited question. $\endgroup$ – Donbeo Oct 1 '15 at 14:00
  • $\begingroup$ The edited question is much harder ;) I assume that $v$ and $x_K$ refer the the same thing ? I don't see the point of the 1/K factor, if you are looking for a correlation, it will not change anything to the final result $\endgroup$ – RUser4512 Oct 1 '15 at 14:09
  • $\begingroup$ 1/K was required to make $cor(x_K, x_i)=1/K$ . $\endgroup$ – Donbeo Oct 1 '15 at 15:02
4
$\begingroup$

This may be cheating a bit, but if we define ‘uncorrelated’ as having a covariance of 0, the answer is yes. Let $X$ and $Y$ both be zero with probability 1. Then

$\mathop{\mathrm{Cov}}(X,Y)= \mathop{\mathrm{E}}(XY)-\mathop{\mathrm{E}}(X)\mathop{\mathrm{E}}(Y)=0-0=0$

while $X+Y=0$, so $X$ and $Y$ are linearly dependent (by your definition).

Though if you require that the correlation is defined, i.e. that the variances of both $X$ and $Y$ are strictly positive, it’s not possible to find variables fulfilling your criteria (see the other answers).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.