7
$\begingroup$

An average of $B$ binomial i.i.d. random variables, each with variance $\sigma^2,$ has variance $\frac{1}{B}\sigma^2.$

If the variables are simply i.d. (identically distributed, but not necessarily independent) with positive pairwise correlation $\rho$, the variance of the average is $$\rho\sigma^2 + \frac{1-\rho}{B}\sigma^2$$ but I don't understand why.

Can somebody provide a proof?

$\endgroup$
  • 4
    $\begingroup$ If you review the derivation of your first result, it will show you how to obtain the second--and it has nothing to do with the shape of the distribution itself. $\endgroup$ – whuber Jan 27 '14 at 20:20
  • $\begingroup$ Since you keep trying to remove the reference to $B$ that I inserted at the beginning of your question, you had better edit it to tell us explicitly what $B$ means. If it is not the count of your variables, your result is in grave doubt. $\endgroup$ – whuber Jan 27 '14 at 20:29
  • $\begingroup$ @whuber, I B is the number of variables. but the point is that, I don't know how to derive the variance of average of dependent but identically distributed variable. could you help me ? $\endgroup$ – user2806363 Jan 27 '14 at 20:33
  • $\begingroup$ How far can you get with the suggestion I initially gave? Do you know how to derive the first result? If not, please investigate the posts found by searching our site on covariance sum: many of them show how to manipulate covariances of linear combinations of random variables. $\endgroup$ – whuber Jan 27 '14 at 20:36
  • $\begingroup$ @whuber, I couldn't derived. $\endgroup$ – user2806363 Jan 27 '14 at 22:27
7
$\begingroup$

As a very general rule, whenever $X = (X_1, \ldots, X_B)$ are random variables with given covariances $\sigma_{ij}=\text{Cov}(X_i,X_j),$ then the covariance of any linear combination $\lambda \cdot X = \lambda_1 X_1 + \cdots + \lambda_B X_B$ is given by the matrix $\Sigma = (\sigma_{ij})$ via

$$\text{Cov}(\lambda X, \lambda X) = \lambda^\prime \Sigma \lambda.$$

The rest is just arithmetic.

In the present case $\sigma_{ij} = \rho\sigma^2$ when $i\ne j$ and otherwise $\sigma_{ii} = \sigma^2 = \left[\rho + (1-\rho)\right]\sigma^2$. That is to say, we may view $\Sigma$ as the sum of two simple matrices: one has $\rho$ in every entry and the other has values of $1-\rho$ on the diagonal and zeros elsewhere. This leads to an efficient calculation, because evidently

$$\Sigma = \sigma^2\left(\rho 1_B 1_B^\prime + (1-\rho)\mathbb{Id}_B \right)$$

where I have written "$1_B$" for the column vector with $B$ $1$'s in it and "$\mathbb{Id}_B$" for the $B$ by $B$ identity matrix. Whence, factoring out the scalars $\sigma^2$, $\rho$, and $1-\rho$ as appropriate, we obtain

$$\eqalign{ \text{Cov}(\lambda X, \lambda X) &= \lambda^\prime \sigma^2\left(\rho 1_B 1_B^\prime + (1-\rho)\mathbb{Id}_B \right)\lambda \\ &= \left(\lambda^\prime 1_B 1_B^\prime \lambda\right) \rho\sigma^2 + \left(\lambda^\prime \mathbb{Id}_B \lambda \right)(1-\rho)\sigma^2. }$$

For the arithmetic mean, $\lambda = (1/B, 1/B, \ldots, 1/B)$ entailing $$\lambda^\prime 1_B 1_B^\prime \lambda = (\lambda^\prime 1_B)^2 = 1^2 = 1$$ and $$\lambda^\prime \mathbb{Id}_B \lambda = 1/B^2 + 1/B^2 + \cdots + 1/B^2 = 1/B,$$ QED.

$\endgroup$
1
$\begingroup$

I think whuber's post gives a nice proof, however there is a possible caveat in the formula in the original post.

Suppose the correlation in the original post is negative, say minus one. Then just looking at the formula it seems that if we just make $B$ sufficiently large, our mean will have a negative variance, which is of course nonsense.

I think the problem here is that we must be sure that our original assumption is sound. If we say the variables forming the mean are identically distributed (say with mean zero and variance one), then we cannot also impose that they have pairwise correlation of minus one.

Since if this would be true, $X_2 = - X_1$ and $ X_3 = -X_2 = X_1 $ hence the correlation between $X_1$ and $X_3$ would be one, contrary to assumption.

Perhaps there is a way to formulate a condition for when the formula actually holds?

Edit: Ok, the original post assumed positive correlation, is it obvious that the formula always works then? Perhaps the implicit condition should simply always be that the problem is consistenly formulated... my guess is that it would be enough that the covariance matrix is positve-semidefinite and symmetric; then such variables exist.

And the matrix in my example above would have a diagonal consisting of ones and all other entries equal to minus one, hence have -1 as an eigenvalue and thus not positive-semidefinite.

$\endgroup$
  • 1
    $\begingroup$ In the symmetric case (all pair correlations equal), the condition becomes (from memory) that $\rho \ge -\frac1{n-1}$. $\endgroup$ – kjetil b halvorsen Dec 17 '15 at 9:51
  • $\begingroup$ Interesting - would be happy to see a reference $\endgroup$ – Christian Dec 17 '15 at 11:58
  • 1
    $\begingroup$ Here is a reference with proof: math.stackexchange.com/questions/1032456/… $\endgroup$ – kjetil b halvorsen Dec 17 '15 at 15:14
  • 1
    $\begingroup$ And here another reference with proof: stats.stackexchange.com/questions/72790/… $\endgroup$ – kjetil b halvorsen Dec 17 '15 at 15:55
  • 1
    $\begingroup$ I think the proofs given above already shows that. Always when $\rho \ge -\frac1{n-1}$ you can construct that covariance matrix, calculate its determinant and see that it is positive. Then you can construct a multinormal random variable with that covariance matrix. $\endgroup$ – kjetil b halvorsen Dec 18 '15 at 10:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.