Suppose we have a random variable $y$ and a collection $(x_1,\dots,x_n)$ of $n$ random variables that are all uncorrelated: $\operatorname{corr}(x_i,x_j) = 0$ $\forall i \ne j$ and that all have the same correlation with $y$: $\operatorname{corr}(y,x_i) = \rho$ $\forall i$. Obviously, they can't all have correlation $1$ or $-1$ with $y$ and be uncorrelated at the same time, so there must be bounds.

This question came up when I was solving a different problem, finding the correlation between $y$ and the linear model $\hat{y} = X(X^\intercal X)^{-1}X^\intercal y$, where $X = (x_1,\dots,x_n)$ and $y,x_i$ are now thought of as $m\ge n$ random samples from the random variables. I found that if the random variables have mean $0$ and variance $1$, the expected value is $\operatorname{corr}(y,\hat{y})=\rho\sqrt{n}$, there's also a more complicated formula for the general case (can post if interested, wanna keep this post short).

This lead me to conjecture that the bounds for $\rho$ are $\left[-\frac{1}{\sqrt{n}},\frac{1}{\sqrt{n}}\right]$, now thinking about proving it for the general case.

  • $\begingroup$ The bounds are $-1/(n-1)$ and $1.$ My answer in the duplicate thread explicitly addresses general $n,$ not just $n=3.$ $\endgroup$ – whuber Jan 18 '20 at 14:58
  • $\begingroup$ It seems like your answer addresses a different problem, though. The answer talks about n variables, all with the same correlation among each other. Here there are n+1 variables, n of them are uncorrelated, and they have a common correlation with the last variable. $\endgroup$ – Chris_77 Jan 19 '20 at 0:06
  • $\begingroup$ Thank you--I had misread your question. Your conjecture can be proved by characterizing the positive-semidefiniteness of the correlation matrix in terms of determinants of its minors. $\endgroup$ – whuber Jan 19 '20 at 16:48
  • $\begingroup$ Thank you, yes, that solves the problem! $\endgroup$ – Chris_77 Jan 22 '20 at 10:44

In terms of $(x_1, x_2, \ldots, x_n, y)$ the correlation matrix is

$$\Sigma_n = \pmatrix{1 & 0 & \cdots & 0 & \rho\\ 0 & 1 & \cdots & 0 & \rho\\ \vdots & \vdots & \ddots & 0 & \rho\\ 0 & 0 & \cdots & 1 & \rho\\ \rho & \rho & \cdots & \rho & 1}.$$

Its principal proper minors are all identity matrices, which have positive determinants. The determinant of $\Sigma_n$ itself can be computed by first subtracting $\rho$ times each of the first $n$ rows from the last row (of rhos), thereby producing a new bottom row of

$$\pmatrix{0 & 0 & \cdots & 0 & 1 - n\rho^2}.$$

This row-reduction produces an upper triangular matrix, whence its determinant is the product of its diagonal entries, equal to

$$|\Sigma_n| = 1 \times 1 \times \cdots \times 1 \times (1 - n\rho^2) = 1 - n\rho^2.$$

This is non-negative if and only if

$$-\frac{1}{\sqrt{n}} \le \rho \le \frac{1}{\sqrt{n}}.\tag{*}$$

Sylvester's Criterion states this is equivalent to positive-semidefiniteness of $\Sigma_n.$

We don't really need to invoke Sylvester's Criterion. All such matrices $\Sigma_n$ do indeed arise as correlation matrices when $(*)$ holds; and otherwise, because $|\Sigma_n|$ becomes negative for any other values of $\rho,$ it would be impossible for $\Sigma_n$ to be a correlation matrix.

Take, for instance, $n+1$ uncorrelated zero-mean, unit-variance random variables $Z_1, Z_2, \ldots, Z_{n+1}.$ Let $\rho$ satisfy $(*)$ and set

$$\lambda = \pm \sqrt{1/\rho^2 - n}$$

to have the same sign as $\rho.$ Set $$X_i=\pm Z_i\text{ for }1\le i\le n$$ (according to the sign of $\rho$) and $$Y=(Z_1+Z_2 + \cdots + Z_n + \lambda Z_{n+1}).$$ Since

$$\operatorname{Cov}(X_i, Y) = \pm 1$$


$$\operatorname{Cov}(Y,Y) = n + \lambda^2,$$

the correlations between the $X_i$ and $Y$ all equal

$$\operatorname{Cor}(X_i, Y) = \frac{\pm 1}{\sqrt{n + \lambda^2}} = \rho$$

as intended.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.