# Is it possible to have a multivariate random distribution with all its random variables (pair-wise) reverse correlated?

Continuing this question, I want to ask if its possible to have a multivariate random distribution having simultaneously all its random variables reverse correlated. I think it is impossible, because when a pair of distribution is reverse correlated the third distribution will be correlated with one of the two and reverse correlated with the other. So there will be a pair with a positive correlation. This question stems from a university project, where the professor asks to create randomly in the n-dim grid anti-correlated data. This is impossible if the n is >2.

• What's wrong with $$\mathcal{N} \left( \vec 0, \begin{bmatrix} 1 & -\frac{1}{10} & -\frac{1}{10} \\ -\frac{1}{10} & 1 & -\frac{1}{10} \\ -\frac{1}{10} & -\frac{1}{10} & 1 \end{bmatrix} \right)?$$ Commented Nov 16, 2023 at 21:49
• @Alexis how is Galen's comment not right? Doesn't his example show a counter example? Commented Nov 16, 2023 at 22:13
• To support @Galen's counterexample, in this answer, it has been shown for any dimension $n$, such an equi-correlation matrix is a valid covariance matrix for all $\rho > -(n - 1)^{-1}$, pick up any negative value in this range. Commented Nov 16, 2023 at 22:17
• I answered this question at stats.stackexchange.com/a/72795/919 by providing an example of multivariate distributions (of any dimension) with negative correlations among all pairs of variables.
– whuber
Commented Nov 17, 2023 at 2:16
• Just to add to the excellent answers below: we must have $\rho\ge-\tfrac{1}{n-1}$, see eg statisticaloddsandends.wordpress.com/2022/09/23/… Commented Nov 17, 2023 at 11:54

Mathematically, as I commented, you can use the equi-correlation matrix

\begin{align*} \Sigma = \begin{bmatrix} 1 & \rho & \cdots & \rho \\ \rho & 1 & \cdots & \rho \\ \vdots & \vdots & \ddots & \vdots \\ \rho & \rho & \cdots & 1 \end{bmatrix} = (1 - \rho)I_{(n)} + \rho ee^\top \tag{1}\label{1} \end{align*} with $$\rho \in (-(n - 1)^{-1}, 0)$$ as a valid example. The proof of $$\Sigma$$ is positive-definite under this condition, hence a valid covariance matrix is contained in this answer. Here $$e$$ stands for an $$n$$-long column vector whose entries are all ones.

Statistically, a natural follow-up question is: for what specific random vector $$\mathbf{X} = (X_1, X_2, \ldots, X_n)$$, its correlation matrix is exactly the $$\Sigma$$ in $$\eqref{1}$$? A quick (and good) answer is that draw $$\mathbf{X}$$ directly from an $$n$$-dimensional multivariate normal distribution with covariance matrix $$\eqref{1}$$. This answer tries to endow $$X_i$$s a more explicit and granular expression.

When $$\rho \geq 0$$, it is easy to verify that \begin{align*} X_i = \sqrt{\rho}Z + \sqrt{1 - \rho}Y_i, \quad i = 1, 2, \ldots, n, \tag{2}\label{2} \end{align*} where $$Z, Y_1, \ldots, Y_n \text{ i.i.d.} \sim N(0, 1)$$, satisfies $$\operatorname{Corr}(X_i, X_j) = \rho$$, $$1 \leq i \neq j \leq n$$. In mathematical finance, $$\eqref{2}$$ are often used to model default risk of $$n$$ companies, where $$Z$$ is interpreted as systemic factor and $$Y_1, \ldots, Y_n$$ are idiosyncratic factors.

When $$\rho < 0$$, it is relatively more difficult to construct equi-correlated $$X_i$$s. However, some idea can still be learned from the form of $$\eqref{2}$$ -- we can still set $$X_i$$ as a linear combination of $$Z$$ and $$Y_i$$, but must also impose some common intercorrelation between $$Y_i$$ and $$Z$$. Specifically, let $$Y_1, \ldots, Y_n \text{ i.i.d. } \sim N(0, 1)$$, $$Z \sim N(0, 1)$$, but $$\operatorname{Cov}(Y_1, Z) = \cdots = \operatorname{Cov}(Y_n, Z) = c$$ for some constant $$c \in (-1, 1)$$ to be determined by $$\rho$$, then set \begin{align*} X_i = Z + Y_i, \quad i = 1, 2, \ldots, n. \tag{3}\label{3} \end{align*} Note that $$\eqref{3}$$ may be viewed as $$n$$ draws from a random effect model with $$Z$$ as main effect and $$Y_i$$ as error. $$c$$ can be determined by solving the equation for $$1 \leq i \neq j \leq n$$: \begin{align*} \rho = \frac{\operatorname{Cov}(X_i, X_j)}{\sqrt{\operatorname{Var}(X_i)\operatorname{Var}(X_j)}} = \frac{1 + 2c}{2 + 2c}, \end{align*} i.e., $$c = \frac{2\rho - 1}{2(1 - \rho)}$$. It should be noticed that as long as $$\rho < \frac{3}{4}$$, we can ensure that $$|c| < 1$$.

In summary, by choosing $$Z, Y_1, \ldots, Y_n$$ such that $$Y_1, \ldots, Y_n \text{ i.i.d. } \sim N(0, 1)$$, $$Z \sim N(0, 1)$$, $$\operatorname{Cov}(Y_i, Z) = \frac{2\rho - 1}{2(1 - \rho)}$$ and setting $$X_i = Y_i + Z$$, $$i = 1, \ldots, n$$, we can render $$\operatorname{Corr}(\mathbf{X}) = \Sigma$$ in $$\eqref{1}$$ for any $$\rho < \frac{3}{4}$$.

Alert readers may be curious now: as the above construction seems imposing no constraint on $$\rho$$ (except $$\rho < \frac{3}{4}$$), where does the aforementioned constraint $$\rho > -(n - 1)^{-1}$$ apply then? Well, this is because the above calculation concerns only with pairwise correlations, whereas in order that an order $$n$$ symmetric matrix to be a covariance matrix, we must ensure that $$\operatorname{Var}(\beta^\top\mathbf{X}) > 0$$ for any vector $$\beta$$, which results in the $$\rho > -(n - 1)^{-1}$$ constraint. In other words, simply specifying $$\eqref{3}$$ without imposing any constraint on $$\rho$$ might lead to some lurking inconsistencies. Therefore, our construction must be placed under the umbrella condition $$\rho \in (-(n - 1)^{-1}, 0)$$.

One last legit question is, how to make sure the structure of $$Z, Y_1, \ldots, Y_n$$ as specified exists?

One obvious construction of $$Z, Y_1, \ldots, Y_n$$ is: \begin{align*} \begin{bmatrix} Z \\ \mathbf{Y} \end{bmatrix} \sim N_{n + 1}\left(0, \begin{bmatrix} 1 & c e^\top \\ ce & I_{(n)} \end{bmatrix}\right), \end{align*} where $$\mathbf{Y} = (Y_1, \ldots, Y_n)^\top$$. In order that the matrix $$\begin{bmatrix} 1 & c e^\top \\ ce & I_{(n)} \end{bmatrix}$$ is a (non-degenerate) covariance matrix, a necessary condition is its determinant must be positive, which requires \begin{align*} \det\left(\begin{bmatrix} 1 & c e^\top \\ ce & I_{(n)} \end{bmatrix}\right) = \det(I_{(n)}) \times (1 - c^2e^\top e) = 1 - nc^2 > 0. \end{align*} i.e., $$c < \frac{1}{\sqrt{n}}$$ (one can show this is also a sufficient condition). Since $$c = \frac{2\rho - 1}{2(1 - \rho)}$$ in our setting, this in turn requires $$\frac{2\rho - 1}{2(1 - \rho)} < \frac{1}{\sqrt{n}}$$, i.e., $$\rho < \frac{\sqrt{n} + 2}{2\sqrt{n} + 2}$$. This, of course, does not conflict with the base constraint $$\rho > -(n - 1)^{-1}$$.

Inspired by statmerkur's excellent multinomial example, the Dirichlet distribution provides another classical multivariate distribution which has all negative pairwise correlations.

• @User1865345 Thanks for your editing. Do you know how to make a block spoiler? For example, how do I hide most of the contents in my "Addendum 1"? I tried adding >! in front of each line but it didn't work out properly. Commented Nov 17, 2023 at 14:00
• See this Meta Math.SE post. Commented Nov 17, 2023 at 14:18
• @User1865345 Thank you very much! It worked now after I removed all linebreaks. Commented Nov 17, 2023 at 14:57
• @Zhanxiong very clearly explained, thank you so much. Also, I want to thank all of you for the concise and perfect answers. Commented Nov 20, 2023 at 10:05

If $$X \mathrel{:=} \left(X_1, \ldots, X_k\right)^\top \sim \mathop{\mathrm{Multinomial}}\left(n, \left(p_1, \ldots, p_k\right)^\top\right)$$ with $$n \in \mathbb N_{\geq 1}, k \in \mathbb N_{\geq 2},$$ we have $$\mathrm{Cov}(X_i, X_j) = - np_ip_j$$ for all $$i,j \in \{1, \ldots , k\} :i \neq j$$.

Thus, if $$p_i \in (0,1)$$ for all $$i \in \{1, \ldots , k\}$$, then all components of $$X$$ are negatively correlated.
This is not particularly surprising as an increase in the value of one component of $$X$$ must result in a decrease in the value of another component for fixed $$n$$.

• This is a natural, concise and classical example! +1 Commented Nov 17, 2023 at 5:27
• Indeed, you can consider the case $n=1$. Let $A_1, A_2, \dots, A_k$ be any collection of disjoint events each with positive probability. Then the indicator functions $I(A_1), \dots, I(A_k)$ are pairwise negatively correlated. Commented Nov 17, 2023 at 15:41

Let X,Y have a negative covariance/correlation, and define a third variable as a linear sum of those two plus independent noise $$\epsilon$$

$$Z = -X -aY + \epsilon$$

Now, this will be inverse correlated with both of them unless the negative covariance between $$X$$ and $$Y$$ cancels one of the negative terms.

$$Cov(Z,X) = Cov(-X-aY, X) = - Var(X) - a Cov(Y,X)$$ $$Cov(Z,Y) = Cov(-X-aY, Y) = - a Var(Y) - Cov(Y,X)$$

The first terms on the right-hand side of the equations in the linear combinations, $$-Var(X)$$ and $$-a Var(Y)$$, are negative.

It would be no surprise when a variable like $$Z$$ is negatively correlated with $$X$$ and $$Y$$ and when $$Cov(Y,X)$$ is small then the linear combinations can be easily seen as negative without much surprise.

Only when $$-Cov(Y,X)$$ is large then it might be possible that $$Z$$ has a positive correlation with $$X$$ and/or $$Y$$

Such positive correlation happens when

$$-a Cov(Y,X)>Var(X)$$ $$-Cov(Y,X) > aVar(Y)$$

or in terms of the correlation $$\rho^2 = Cov(Y,X)^2/Var(X)Var(Y)$$

$$-\rho > \frac{1}{a} \frac{Var(X)}{Var(Y)}$$ $$-\rho > a \frac{Var(Y)}{Var(X)}$$

Code example

set.seed(1)
n = 100

X = rnorm(n)
Y = -0.5 * X + rnorm(n)
Z = -X - 0.5*Y + rnorm(n)

M = cbind(X,Y,Z)
cov(M)


gives

           X          Y          Z
X  0.8067621 -0.4042365 -0.5875672
Y -0.4042365  1.1200783 -0.2134165
Z -0.5875672 -0.2134165  1.7757109

• +1 Beautiful! Helps me build an understanding on top of Dave's empirical demonstration. Commented Nov 16, 2023 at 22:37

YES

(I find this fact surprising, too.)

library(MASS)
set.seed(2023)
R <- 1000
r1 <- runif(R, -1, 0)
r2 <- runif(R, -1, 0)
r3 <- runif(R, -1, 0)
signs <- rep(NA, R)
for (i in 1:R){
S <- matrix(
c(
1, r1[i], r3[i],
r1[i], 1, r2[i],
r3[i], r2[i], 1
), 3, 3
)
signs[i] <- sign(min(eigen(S)$values)) # print(i) } i <- which(signs == 1)[1] S <- matrix( c( 1, r1[i], r3[i], r1[i], 1, r2[i], r3[i], r2[i], 1 ), 3, 3 ) X <- MASS::mvrnorm(1000, rep(0, 3), S) cor(X)  Here, I loop many times until I find a candidate correlation matrix with off-diagonal elements less than zero that has a smallest eigenvalue greater than zero, meaning that this is a valid correlation matrix with all margins negatively correlated with each other. I also simulate from a multivariate normal distribution with such a covariance matrix and show the empirical correlation values to be less than zero. (This can be seen as a linear algebra problem asking if a symmetric matrix can have positive diagonal elements, negative diagonal elements, and positive eigenvalues. If you can prove that to exist (the above code gives such a matrix), then you get the desired multivariate distribution as a multivariate normal with that matrix as the covariance matrix.) • +1 Very cool! I share your surprise. :) Commented Nov 16, 2023 at 21:58 • I don't see the surprise and why you need to do these simulations. Isn't the comment by Galen sufficient. Or else just do X = rnorm(n);Y = -0.5 * X + rnorm(n);Z = -X - 0.5*Y + rnorm(n) Commented Nov 16, 2023 at 22:13 • A few moments contemplating a regular tetrahedron should remove all sense of surprise, because all examples of this phenomenon concern vectors that approximate the vertices of a tetrahedron (in any finite number of dimensions). For two variables that "tetrahedron" is a zero-centered line segment in$\mathbb R^1;$for three variables it is a zero-centered equilateral triangle in$\mathbb R^2;$for four variables it is the usual (zero-centered) Platonic solid in$\mathbb R^3;\$ and so on. We're concerned with the angles made between the rays through the vertices: something you can literally see
– whuber
Commented Nov 21, 2023 at 17:18