Series of mutually independent variables that are dependent on an auxiliary random variable

Question

Suppose that $X_1, X_2, ... , X_n$ are mutually independent random variables. There is a random variable $C \sim U(-1,1)$, which all $X$s depend on. How can I construct such $X$s so that they are unconditionally independent with zero mean?

Answer 1

If you want an actual example then it is easier to construct the $X_i$ first and then create $C$ depending on them.

For example, you could have $X_i \sim N(0,1)$ independently and have $C=2\Phi^{-1}\left(\frac1{\sqrt{n}}\sum\limits_{i=1}^n X_i\right)-1$ where $\Phi^{-1}$ is the quantile function of a standard normal distribution.

In R, to get a matrix of $n$ columns and $k$ rows (a row for each set of observations of $X_1,X_2,\ldots, X_n$), you could use:

Xmat <- matrix(rnorm(n*k), ncol=n)
C <- 2 * pnorm(rowSums(Xmat) / sqrt(n))

It is possible to do this backwards, starting with $k$ values of $C$, but is more complicated as you have to condition each $X_j$ not only on $C$ but also on the previously observed $X$ values.

C <- runif(k, -1, 1)
Xmat <- matrix(nrow=k, ncol=n)
S <- sqrt(n) * qnorm((C+1)/2)
for (j in n:1){
  Xmat[,j] <- rnorm(k, S/j, sqrt(1-1/j))
  S <- S - Xmat[,j]
  }

Either way, you will get $C\sim U(-1,1)$ and a positive correlation between each $X_i$ and $C$, while keeping the $X_i$ unconditionally mutually independent.

Conditioned on $C=c\in (-1,1)$, the $X_i$ and $X_j$ are not independent; they each have a conditional variance of $\frac{n-1}{n}$ and a conditional covariance between them of $-\frac{1}{n}$ and so a conditional correlation between them of $-\frac{1}{n-1}$. Unconditionally, their variances are $1$ and their covariances and correlations are $0$.

Answer 2

Let $\{Y_i, i=1,2,\ldots, n\}$ be any finite collection of random variables for which

(i) $\{C\} \bigcup \{Y_i\}$ are independent and

(ii) $E[|Y_i|] \lt \infty$ for all $i.$

For $i=1, 2, \ldots,$ let $b_i:[0,1)\to \{0,1\}$ be the $i^\text{th}$ bit in the binary expansion of its argument (choosing, say, $b_i(x) = 0$ when there is more than one possible expansion, so that this is well-defined).

Define $B_i = 2*b_i(|C|) - 1$ and notice that the $B_i$ are iid Rademacher variables (their values are $\pm1$ with equal probability).

The collection $X_i = B_iY_i$ does the trick:

• The $X_i$ are independent because they are functions of the independent variables $(B_i, Y_i).$

• Therefore $E[X_i] = E[B_iY_i] = E[B_i]E[Y_i] = (0)E[Y_i] = 0$ because $E[Y_i]$ is finite.

Here, to show how $C$ controls the distribution, is an illustration of a thousand realizations of $(X_1,X_2,X_3)$ using this method, starting from distributions named in the plot:

This is the R code to produce such illustrations:

to.binary <- Vectorize(function(u, ndigits = 52) {
  bases <- 2 ^ seq_len(ndigits)
  2 * c(floor((bases * u) %% 2)) - 1 # Converts 0 and 1 to -1 and 1, resp.
}, "u")
#
# Simulate `Y` and `C` then create `X` from them.
#
n.sim <- 1e3
Y <- list(Normal = rnorm, Uniform = runif, Exponential = rexp) # The RNGs
set.seed(17)
Y.realized <- sapply(Y, \(f) f(n.sim))
C <- runif(n.sim, -1, 1)
#
# Here's the algorithm:
#
B <- t(to.binary(C, length(Y)))
X.realized <- Y.realized * B
#
# Plot the results.
#
colnames(X.realized) <- names(Y)
pairs(cbind(X.realized, C), col = "#00000040")