If we know that $X_1, X_2, ..., X_n \sim \text{IID N}(\mu, \sigma^2)$, what is $\mathbb{Cov}(X_1, \bar{X}_n)$? Because the set of variables follows an IID distribution, would the covariance just be zero?


The covariance here will not be zero because the sample mean includes the variable $X_1$ as part of it; a higher values of the latter variable is going to give you a higher sample mean and a lower value is going to give you a lower sample mean. Consequently, we would expect these two things to be positively correlated.

Now, using the fact that the covariance operator is linear in each of its arguments, you get:

$$\begin{align} \mathbb{C}(X_1, \bar{X}_n) &= \mathbb{C} \Bigg( X_1, \frac{1}{n} \sum_{i=1}^n X_i \Bigg) \\[6pt] &= \frac{1}{n} \sum_{i=1}^n \mathbb{C} ( X_1, X_i ) \\[6pt] &= \frac{1}{n} \mathbb{C}(X_1, X_1) \\[6pt] &= \frac{1}{n} \mathbb{V}(X_1) \\[6pt] &= \frac{\sigma^2}{n}. \\[6pt] \end{align}$$

This confirms that the covariance is positive so long as $\sigma>0$. As $n \rightarrow \infty$ the sample mean depends less and less on the value $X_1$, and the covariance diminishes to zero. (Note also that this result does not depend on the assumption of normality of the underlying variables; it holds for any underlying distribution with finite variance.)

  • 1
    $\begingroup$ (+1). I think I'm having an intellectual blackout: How do you get from the second line of the equation to the third? I'm suspecting because of the IID assumption? $\endgroup$ – COOLSerdash Apr 14 at 15:49
  • 1
    $\begingroup$ @COOLSerdash: Since the data values are IID they are uncorrelated, so you have $\mathbb{C}(X_1, X_i) = 0$ for all $i \neq 1$. Consequently, the only non-zero term in the sum is $\mathbb{C}(X_1, X_1)$. $\endgroup$ – Ben Apr 15 at 1:55
  • $\begingroup$ Thanks Ben, I suspected it was something easy ... $\endgroup$ – COOLSerdash Apr 15 at 9:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.