Covariance between individual IID variables and their sample mean

Question

If we know that $X_1, X_2, ..., X_n \sim \text{IID N}(\mu, \sigma^2)$, what is $\mathbb{Cov}(X_1, \bar{X}_n)$? Because the set of variables follows an IID distribution, would the covariance just be zero?

Answer 1

The covariance here will not be zero because the sample mean includes the variable $X_1$ as part of it; a higher values of the latter variable is going to give you a higher sample mean and a lower value is going to give you a lower sample mean. Consequently, we would expect these two things to be positively correlated.

Now, using the fact that the covariance operator is linear in each of its arguments, you get:

$$\begin{align} \mathbb{C}(X_1, \bar{X}_n) &= \mathbb{C} \Bigg( X_1, \frac{1}{n} \sum_{i=1}^n X_i \Bigg) \\[6pt] &= \frac{1}{n} \sum_{i=1}^n \mathbb{C} ( X_1, X_i ) \\[6pt] &= \frac{1}{n} \mathbb{C}(X_1, X_1) \\[6pt] &= \frac{1}{n} \mathbb{V}(X_1) \\[6pt] &= \frac{\sigma^2}{n}. \\[6pt] \end{align}$$

This confirms that the covariance is positive so long as $\sigma>0$. As $n \rightarrow \infty$ the sample mean depends less and less on the value $X_1$, and the covariance diminishes to zero. (Note also that this result does not depend on the assumption of normality of the underlying variables; it holds for any underlying distribution with finite variance.)