Set of uncorrelated but linearly dependent variables

Question

Is it possible to have a set of $K$ variables that are uncorrelated but linearly dependent?

i.e. $cor(x_i, x_j)=0$ and $ \sum_{i=1}^K a_ix_i=0$

If yes can you write an example ?

EDIT: From the answers it follows that it is not possible.

Would it at least be possible that $\mathbb{P}(|\hat \rho_{x_i, x_j}-\hat \rho_{x_i, v}|<\epsilon)$ where $\hat\rho$ is the estimated correlation coefficient estimated from $n$ samples of the variables and $v$ is a variable that is uncorrelated with $x_i$.

I am thinking something like $x_K=\dfrac{1}{K} \sum_{i=1}^{K-1} x_i$ $K>>0$

Answer 1

As @RUser4512's answer shows, uncorrelated random variables cannot be linearly dependent. But, nearly uncorrelated random variables can be linearly dependent, and one example of these is something dear to the statistician's heart.

Suppose that $\{X_i\}_{i=1}^K$ is a set of $K$ uncorrelated unit-variance random variables with common mean $\mu$. Define $Y_i = X_i - \bar{X}$ where $\bar{X} = \frac 1K \sum_{i=1}^K X_i$. Then, the $Y_i$ are zero-mean random variables such that $\sum_{i=1}^K Y_i = 0$, that is, they are linearly dependent. Now, $$Y_i = \frac{K-1}{K} X_i - \frac 1K\sum_{j \neq i}X_j$$ so that $$\operatorname{var}(Y_i) = \left(\frac{K-1}{K}\right)^2+\frac{K-1}{K^2} = \frac{K-1}{K}$$ while $$\operatorname{cov}(Y_i,Y_j) = -2\left(\frac{K-1}{K}\right)\frac 1K + \frac{K-2}{K^2}= \frac{-1}{K}$$ showing that the $Y_i$ are nearly uncorrelated random variables with correlation coefficient $\displaystyle -\frac{1}{K-1}$.

See also this earlier answer of mine.

Answer 2

No.

Suppose one of the $a_i$ is non zero. Without loss of generality, let's assume $a_1=1$.

For $K=2$, this implies $x_1=-a_2x_2$ and $cor(x_1,x_2)=-1$. But this correlation is zero. $a_1$ must be zero as well, contradicting the existence of a linear relationship.

For any $K$, $x_1=-\sum_{i>1}a_ix_i$ and $cor(x_1,x_k)=-1$. But, by you hypothesis, $cor(x_1,x_k)=0$. The $a_i$'s are zero (for $i>1$) and so must be $a_1$.

Answer 3

This may be cheating a bit, but if we define ‘uncorrelated’ as having a covariance of 0, the answer is yes. Let $X$ and $Y$ both be zero with probability 1. Then

$\mathop{\mathrm{Cov}}(X,Y)= \mathop{\mathrm{E}}(XY)-\mathop{\mathrm{E}}(X)\mathop{\mathrm{E}}(Y)=0-0=0$

while $X+Y=0$, so $X$ and $Y$ are linearly dependent (by your definition).

Though if you require that the correlation is defined, i.e. that the variances of both $X$ and $Y$ are strictly positive, it’s not possible to find variables fulfilling your criteria (see the other answers).