3
$\begingroup$

Question.

Let $X_1, X_2, ..., X_n$ be a set of normal random variables, each with variance ${\sigma }^2$ and mean 0. For each $i,j$ in pair in $X$, $Cov(X_i,X_j)=V$.

Further, let $Y_1, Y_2, ..., Y_m$ be a second set of random variables, where each $Y_k=\displaystyle \sum_{i=1}^n w_{i,k}X_i$. Each $w_{i,k}$ is an independent uniform random variable on $[0, 1]$.

What is the variance of $Z=\displaystyle \sum_{k=1}^mY$?


My progress/observations.

The variance of $w_{i,k}X_i$ is the product of a uniform random variable and a normal random variable. I believe this can be calculated similarly to, for example, this answer.

$Y_k$ individually is a sum of $w_{i,k}X_i$. I believe this can be thought of as $Var(w_{1,k}X_1)+ ...+ Var(w_{n,k}X_n) + 2*(Cov(w_{1,k}X_1,w_{2,k}X_2)+ Cov(w_{1,k}X_1,w_{3,k}X_3)+...+Cov(w_{n-1,k}X_1,w_{n,k}X_2))$

Simulations show that $Var(Z)$ scales roughly linearly with $n$.

$\endgroup$

1 Answer 1

6
$\begingroup$

First, note that $$ \begin{align} Z&=\sum_{k=1}^m Y_k \\&= \sum_{k=1}^m\sum_{i=1}^n w_{i,k}X_i \\&= \sum_{i=1}^n(\sum_{k=1}^m w_{i,k})X_i \\&= \sum_{i=1}^n w_i X_i \\&=\mathbf{w}^T\mathbf{X} \end{align} $$ where $w_i=\sum_{k=1}^m w_{i,k}$. Clearly $E(w_i)=m/2$ and $\operatorname{Var}w_i=m/12$.

Using the law of total variance, and this formula for the expected value of a quadratic form, $$ \begin{align} \operatorname{Var}Z &= \operatorname{Var}(\mathbf{w}^T\mathbf{X}) \\&= E\operatorname{Var}(\mathbf{w}^T\mathbf{X}|\mathbf{w})+\operatorname{Var}E(\mathbf{w}^T\mathbf{X}|\mathbf{w}) \\&=E(\mathbf{w}^T\operatorname{Var}(\mathbf{X})\mathbf{w})+\operatorname{Var}(\mathbf{w}^TE(\mathbf{X})) \\&=\operatorname{tr}(\operatorname{Var}(\mathbf{X})\operatorname{Var}(\mathbf{w}))+E(\mathbf{w})^T\operatorname{Var}(\mathbf{X}) E(\mathbf{w})+E(\mathbf{X})^T\operatorname{Var}(\mathbf{w}) E(\mathbf{X}) \\&=\frac{nm\sigma^2}{12}+\frac{nm^2(\sigma^2 +(n-1)V)}{2^2}. \end{align} $$

The following simulation suggests that this is correct:

n <- 5
m <- 7
sigma2 <- 3
V <- 1
Sigma <- matrix(V,n,n)
diag(Sigma) <- sigma2

Z <- replicate(1e+5, {
X <- as.vector(mvtnorm::rmvnorm(1, sigma = Sigma))
w <- matrix(runif(m*n),m,n)
Y <- w %*% X
sum(Y)
})

var(Z)
#> [1] 436.8009
n*m*sigma2/12 + n*m^2*(sigma2 + (n-1)*V)/4
#> [1] 437.5
$\endgroup$
3
  • $\begingroup$ I was trying to make an attempt to follow your derivation of Var(Z), but being lazy I skipped to the last line, and I noticed the dependency of V and the case when V=0. Wouldn't we expect in that case that the second term is zero? $\text{Var } Z = nm\sigma^2/12$ $\endgroup$ Sep 11, 2023 at 14:56
  • 1
    $\begingroup$ @SextusEmpiricus I think you would still have the second term. In that simpler case, you would still need the variance of $w_iX_i$ which involves the squared expectations of the the $w_i$'s, see e.g. en.wikipedia.org/wiki/… that appears as $m^2/4$ in the second term. $\endgroup$ Sep 11, 2023 at 15:05
  • $\begingroup$ @JarloTufto ah, I was confused with the formula $E[XY] = E[X]E[Y]$ but for the variance you get multiple terms. $\endgroup$ Sep 11, 2023 at 16:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.