# Distribution of a sum of linear combinations of random variables, each drawn from a set of random variables

Question.

Let $$X_1, X_2, ..., X_n$$ be a set of normal random variables, each with variance $${\sigma }^2$$ and mean 0. For each $$i,j$$ in pair in $$X$$, $$Cov(X_i,X_j)=V$$.

Further, let $$Y_1, Y_2, ..., Y_m$$ be a second set of random variables, where each $$Y_k=\displaystyle \sum_{i=1}^n w_{i,k}X_i$$. Each $$w_{i,k}$$ is an independent uniform random variable on $$[0, 1]$$.

What is the variance of $$Z=\displaystyle \sum_{k=1}^mY$$?

My progress/observations.

The variance of $$w_{i,k}X_i$$ is the product of a uniform random variable and a normal random variable. I believe this can be calculated similarly to, for example, this answer.

$$Y_k$$ individually is a sum of $$w_{i,k}X_i$$. I believe this can be thought of as $$Var(w_{1,k}X_1)+ ...+ Var(w_{n,k}X_n) + 2*(Cov(w_{1,k}X_1,w_{2,k}X_2)+ Cov(w_{1,k}X_1,w_{3,k}X_3)+...+Cov(w_{n-1,k}X_1,w_{n,k}X_2))$$

Simulations show that $$Var(Z)$$ scales roughly linearly with $$n$$.

First, note that \begin{align} Z&=\sum_{k=1}^m Y_k \\&= \sum_{k=1}^m\sum_{i=1}^n w_{i,k}X_i \\&= \sum_{i=1}^n(\sum_{k=1}^m w_{i,k})X_i \\&= \sum_{i=1}^n w_i X_i \\&=\mathbf{w}^T\mathbf{X} \end{align} where $$w_i=\sum_{k=1}^m w_{i,k}$$. Clearly $$E(w_i)=m/2$$ and $$\operatorname{Var}w_i=m/12$$.

Using the law of total variance, and this formula for the expected value of a quadratic form, \begin{align} \operatorname{Var}Z &= \operatorname{Var}(\mathbf{w}^T\mathbf{X}) \\&= E\operatorname{Var}(\mathbf{w}^T\mathbf{X}|\mathbf{w})+\operatorname{Var}E(\mathbf{w}^T\mathbf{X}|\mathbf{w}) \\&=E(\mathbf{w}^T\operatorname{Var}(\mathbf{X})\mathbf{w})+\operatorname{Var}(\mathbf{w}^TE(\mathbf{X})) \\&=\operatorname{tr}(\operatorname{Var}(\mathbf{X})\operatorname{Var}(\mathbf{w}))+E(\mathbf{w})^T\operatorname{Var}(\mathbf{X}) E(\mathbf{w})+E(\mathbf{X})^T\operatorname{Var}(\mathbf{w}) E(\mathbf{X}) \\&=\frac{nm\sigma^2}{12}+\frac{nm^2(\sigma^2 +(n-1)V)}{2^2}. \end{align}

The following simulation suggests that this is correct:

n <- 5
m <- 7
sigma2 <- 3
V <- 1
Sigma <- matrix(V,n,n)
diag(Sigma) <- sigma2

Z <- replicate(1e+5, {
X <- as.vector(mvtnorm::rmvnorm(1, sigma = Sigma))
w <- matrix(runif(m*n),m,n)
Y <- w %*% X
sum(Y)
})

var(Z)
#> [1] 436.8009
n*m*sigma2/12 + n*m^2*(sigma2 + (n-1)*V)/4
#> [1] 437.5

• I was trying to make an attempt to follow your derivation of Var(Z), but being lazy I skipped to the last line, and I noticed the dependency of V and the case when V=0. Wouldn't we expect in that case that the second term is zero? $\text{Var } Z = nm\sigma^2/12$ Sep 11, 2023 at 14:56
• @SextusEmpiricus I think you would still have the second term. In that simpler case, you would still need the variance of $w_iX_i$ which involves the squared expectations of the the $w_i$'s, see e.g. en.wikipedia.org/wiki/… that appears as $m^2/4$ in the second term. Sep 11, 2023 at 15:05
• @JarloTufto ah, I was confused with the formula $E[XY] = E[X]E[Y]$ but for the variance you get multiple terms. Sep 11, 2023 at 16:06