I have a random variable denoted $N$. Then I have random variables denoted $X_1$, $X_2$, ..., $X_N$ distributed according to a uniform distribution. I have also random variables denoted $Y_1$, $Y_2$, ..., $Y_N$ distributed according to this same probability distribution.

If we consider the variable $S_X$ such that $S_X=\sum_{i=1}^{N}X_i$ and $S_Y$ such that $S_Y=\sum_{i=1}^{N}Y_i$.

Are $S_X$ and $S_Y$ mutually independent ?

If we extend this problem to the case of any number $A$ of sums, for instance $S_X$, $S_Y$ and $S_Z$ when $A=3$. Are these $A$ variables mutually independent ?

Thank you very much.

  • 2
    $\begingroup$ You might like to take a look at the law of total covariance, which in your case is: $\text{cov}(S_X,S_Y)=\text{E}(\text{cov}(S_X,S_Y \mid N))+\text{cov}(\text{E}(S_X\mid N),\text{E}(S_Y\mid N))$. Pay particular attention to the second term, which is a covariance of two functions of $N$. $\endgroup$
    – Glen_b
    Jul 11 '13 at 5:51

Even assuming the $X_i$ and $Y_j$ are independent of $N$ and of one another, $S_X$ and $S_Y$ will be positively correlated and (therefore) not independent. We can calculate that correlation given the distribution of $N$.

To illustrate, let $N$ take on either the value $1$ or $12$ with equal probability. Then half the time $(S_X, S_Y)$ is a point in the unit square (when $N=1$) and half the time it is a point near $(6,6)$ and--according to the Central Limit Theorem--is dispersed randomly around that location in an approximately bivariate Normal manner.


This scatterplot depicts a simulation of 10,000 independent $(S_X, S_Y)$ pairs. The colors distinguish the values of $N$.

The correlation should be obvious in this plot: $S_X$ and $S_Y$ both tend to be small when $N$ is small and large when $N$ is large, whence they tend to be small together or large together: that's positive correlation.

The correlation can be computed using formulas for nested (iterated) expectations. For example,

$$\mathbb{E}_{N;X_i}[S_X] = \mathbb{E}_N[\mathbb{E}_{X_i|N}[S_X | N]] = (1/2 + 12(1/2))/2 = 13/4.$$

In a similar manner all relevant multivariate moments of $(S_X, S_Y)$ can be computed based on knowing that $\mathbb{E}[X_i] = \mathbb{E}[Y_j] = 1/2$ and $\mathbb{E}[X_i^2] = \mathbb{E}[Y_j^2] = 1/3$ (if we assume the $X_i$ and $Y_j$ are all independent). The variances are $56/3 - (13/4)^2 \approx 8.104$ and the covariance is $145/8 - (13/4)^2 = 7.5625$, whence the correlation is $363/389 \approx 0.9332$. Indeed, in this simulation the observed correlation was $0.9316$, apparently differing from this theoretical value only by chance variation.

This answer obviously extends to more than two such sums. It provides a nice example of variables that can be conditionally independent (which will be the case when the $X_i$ are independent of the $Y_j$) but not themselves independent.

Simulation Code

The simulation was carried out in R:

N <- 10^4
m <- 12
n <- ifelse(runif(N) < 1/2, 1, m)
x <- matrix(runif(m*N), ncol=m)
y <- matrix(runif(m*N), ncol=m)

s <- t(sapply(1:N, function(i) c(sum(x[i, 1:n[i]]), sum(y[i, 1:n[i]]))))
col = ifelse(n==1, "Blue", "Red")
plot(s, col=col, pch=19, cex=.5, xlab="X", ylab="Y")
  • $\begingroup$ Thank you very much for this simulation. It's more than I could expect. $\endgroup$
    – Dingo13
    Jul 10 '13 at 18:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.