Calculating the variance of a weight-average when the weights also have a variance

Question

Assume there is a series of random variables $X_1$, $X_2$, ..., $X_N$ representing a series of values to be weight-averaged, and a corresponding series of random variables $W_1$, $W_2$, ..., $W_N$ representing the weights themselves.

Let us define the random variable $Z$ to be the result of the weight-average, that is:

$Z = \frac{\sum_i X_i W_i}{\sum_i W_i}$

If we assume that all the random variables are independent, and we know that expected value and variance of all the random variables (other than $Z$), is it possible to calculate the expected value and variance of $Z$ analytically?

My intuition tells me that $E[Z] = \frac{\sum_i E[X_i] E[W_i]}{\sum_i E[W_i]}$, and I have run experiments that suggest this is the case, but I'm unable to prove it.

Is there an analytic solution to calculating $VAR[Z]$ and $E[Z]$?

Answer 1

The intuitive formula is not generally correct. I will put off presenting a counterexample until we have performed enough analysis to find a simple one.

---

Notice

$$\frac{\sum_i X_i W_i}{\sum_j W_j} = \sum_i X_i \left(\frac{W_i}{\sum_j W_j}\right).$$

Write $S_W = \sum W_i$ for the sum of the weights. Linearity of expectation and independence of $X_i$ from the $W_j$ imply

$$E\left[\frac{\sum_i X_i W_i}{\sum_j W_j}\right] = \sum_i E\left[X_i \left(\frac{W_i}{S_W}\right)\right] = \sum_i E[X_i]E\left[\frac{W_i}{S_W}\right].$$

That is as far as you can generally take it.

However, often weights sum to unity: that will make $S_W$ disappear from the final expression. Moreover, in many cases the $W_i$ are identically distributed. No matter whether $S_W$ is constant or not, for any $i$ we find

$$1 = E[1] = E\left[\frac{W_1+W_2+\cdots+W_n}{W_1+W_2+\cdots+W_n}\right] = E\left[\frac{W_1}{S_W} + \frac{W_2}{S_W} + \cdots + \frac{W_n}{S_W}\right] = n E\left[\frac{W_i}{S_W}\right],$$

showing that

$$E\left[\frac{W_i}{S_W}\right] = \frac{1}{n},\ i = 1,2,\ldots, n.$$

In effect, when the weights are identically distributed, then whether they are independent or not, the expectation of the weighted mean of the $X_i$ is the mean of the expectations of the $X_i$ (provided only that the $X_i$ are independent of the $W_j$).

Another special case occurs when the $X_i$ have a common mean, say $\mu$ (which occurs when the $X_i$ are iid). Now the expectation formula reduces to

$$\sum_i E[X_i]E\left[\frac{W_i}{S_W}\right] = \mu\sum_i E\left[\frac{W_i}{S_W}\right] = \mu E\left[\frac{\sum_i W_i}{S_W}\right] = \mu E[1] = \mu.$$

---

You could try a similar approach with the variance, which requires computing the expectation of the square of the original fraction. Writing

$$\omega_i = \frac{W_i}{\left(\sum_j W_j\right)^2},$$

you will find (upon expanding the square of the sum into a double sum of products) that

$$E\left[\left(\frac{\sum_i X_i W_i}{\sum_j W_j} \right)^2\right] = \sum_{i,j} E[X_iX_j]\, E[\omega_i\omega_j].$$

This is a weighted sum of the expectations $E[X_iX_j]$ that appear in computing the variance of $X_1+X_2+\cdots+X_n.$ However, no simplification is generally possible unless somehow the weights $E[\omega_i\omega_j]$ simplify, such as all being equal (which happens when the weights are iid -- and this time we need them to be uncorrelated). You have seen enough of the relevant algebra here to be able to carry it out yourself in such special cases.

---

We're ready for a counterexample. The smallest possible one will involve four independent variables $X_1,X_2,W_1,W_2;$ the $X_i$ should not have the same means; and (for similar reasons) the $W_i$ should not have the same means.

Consider, then, binary variables with equal probabilities of each value. Let the possible values of $X_1$ be $\{0,1\},$ of $X_2$ be $\{0,2\},$ of $W_1$ be $\{1-\alpha,\alpha\},$ and of $W_2$ be $\{1-\beta,\beta\}$ for $0\lt \alpha,\beta\le 1/2.$ To make the $W_i$ as different as possible, let's select two extreme values; say, $\alpha\approx 0$ and $\beta = 1/2.$

The calculation is straightforward. The intuitive formula in the question, $E[Z] \overset{?}{=} \frac{\sum_i E[X_i] E[W_i]}{\sum_i E[W_i]},$ gives $3/4,$ but in fact $E[Z]\approx 5/6 \ne 3/4.$ Here is R code to carry it out.

a <- 1e-15
b <- 1/2
X <- expand.grid(X1=c(0,1), X2=c(0,2), W1=c(1-a,a), W2=c(1-b,b))
X$Z = with(X, (X1 * W1 + X2 * W2) / (W1 + W2))
E.z <- with(X, mean(Z))
F.z <- with(X, (mean(X1) * mean(W1) + mean(X2) * mean(W2)) / (mean(W1) + mean(W2)))
(c(Expectation = E.z, `Formula?` = F.z))

Its output is

Expectation    Formula? 
  0.8333333   0.7500000

Answer 2

If your two random variables $W_i$ and $X_i$ are independent, then from the law of total expectation the expectation of the product of two random variables is the product of their expectations, eg $E[X_i W_i] = E[X_i] E[W_i]$.

As for the Variance, this is defined in terms of the square of the variable, eg:

$$ \text{Var}(Z_i) = E[Z_i^2] - E[Z_i]^2 $$