# Bounds for common correlation of uncorrelated random variables with another random variable

Suppose we have a random variable $$y$$ and a collection $$(x_1,\dots,x_n)$$ of $$n$$ random variables that are all uncorrelated: $$\operatorname{corr}(x_i,x_j) = 0$$ $$\forall i \ne j$$ and that all have the same correlation with $$y$$: $$\operatorname{corr}(y,x_i) = \rho$$ $$\forall i$$. Obviously, they can't all have correlation $$1$$ or $$-1$$ with $$y$$ and be uncorrelated at the same time, so there must be bounds.

This question came up when I was solving a different problem, finding the correlation between $$y$$ and the linear model $$\hat{y} = X(X^\intercal X)^{-1}X^\intercal y$$, where $$X = (x_1,\dots,x_n)$$ and $$y,x_i$$ are now thought of as $$m\ge n$$ random samples from the random variables. I found that if the random variables have mean $$0$$ and variance $$1$$, the expected value is $$\operatorname{corr}(y,\hat{y})=\rho\sqrt{n}$$, there's also a more complicated formula for the general case (can post if interested, wanna keep this post short).

This lead me to conjecture that the bounds for $$\rho$$ are $$\left[-\frac{1}{\sqrt{n}},\frac{1}{\sqrt{n}}\right]$$, now thinking about proving it for the general case.

• The bounds are $-1/(n-1)$ and $1.$ My answer in the duplicate thread explicitly addresses general $n,$ not just $n=3.$ – whuber Jan 18 '20 at 14:58
• It seems like your answer addresses a different problem, though. The answer talks about n variables, all with the same correlation among each other. Here there are n+1 variables, n of them are uncorrelated, and they have a common correlation with the last variable. – Chris_77 Jan 19 '20 at 0:06
• Thank you--I had misread your question. Your conjecture can be proved by characterizing the positive-semidefiniteness of the correlation matrix in terms of determinants of its minors. – whuber Jan 19 '20 at 16:48
• Thank you, yes, that solves the problem! – Chris_77 Jan 22 '20 at 10:44

In terms of $$(x_1, x_2, \ldots, x_n, y)$$ the correlation matrix is

$$\Sigma_n = \pmatrix{1 & 0 & \cdots & 0 & \rho\\ 0 & 1 & \cdots & 0 & \rho\\ \vdots & \vdots & \ddots & 0 & \rho\\ 0 & 0 & \cdots & 1 & \rho\\ \rho & \rho & \cdots & \rho & 1}.$$

Its principal proper minors are all identity matrices, which have positive determinants. The determinant of $$\Sigma_n$$ itself can be computed by first subtracting $$\rho$$ times each of the first $$n$$ rows from the last row (of rhos), thereby producing a new bottom row of

$$\pmatrix{0 & 0 & \cdots & 0 & 1 - n\rho^2}.$$

This row-reduction produces an upper triangular matrix, whence its determinant is the product of its diagonal entries, equal to

$$|\Sigma_n| = 1 \times 1 \times \cdots \times 1 \times (1 - n\rho^2) = 1 - n\rho^2.$$

This is non-negative if and only if

$$-\frac{1}{\sqrt{n}} \le \rho \le \frac{1}{\sqrt{n}}.\tag{*}$$

Sylvester's Criterion states this is equivalent to positive-semidefiniteness of $$\Sigma_n.$$

We don't really need to invoke Sylvester's Criterion. All such matrices $$\Sigma_n$$ do indeed arise as correlation matrices when $$(*)$$ holds; and otherwise, because $$|\Sigma_n|$$ becomes negative for any other values of $$\rho,$$ it would be impossible for $$\Sigma_n$$ to be a correlation matrix.

Take, for instance, $$n+1$$ uncorrelated zero-mean, unit-variance random variables $$Z_1, Z_2, \ldots, Z_{n+1}.$$ Let $$\rho$$ satisfy $$(*)$$ and set

$$\lambda = \pm \sqrt{1/\rho^2 - n}$$

to have the same sign as $$\rho.$$ Set $$X_i=\pm Z_i\text{ for }1\le i\le n$$ (according to the sign of $$\rho$$) and $$Y=(Z_1+Z_2 + \cdots + Z_n + \lambda Z_{n+1}).$$ Since

$$\operatorname{Cov}(X_i, Y) = \pm 1$$

and

$$\operatorname{Cov}(Y,Y) = n + \lambda^2,$$

the correlations between the $$X_i$$ and $$Y$$ all equal

$$\operatorname{Cor}(X_i, Y) = \frac{\pm 1}{\sqrt{n + \lambda^2}} = \rho$$

as intended.