2
$\begingroup$

If we know that $X_1, X_2, ..., X_n \sim \text{IID N}(\mu, \sigma^2)$, what is $\mathbb{Cov}(X_1, \bar{X}_n)$? Because the set of variables follows an IID distribution, would the covariance just be zero?

$\endgroup$

1 Answer 1

1
$\begingroup$

The covariance here will not be zero because the sample mean includes the variable $X_1$ as part of it; a higher values of the latter variable is going to give you a higher sample mean and a lower value is going to give you a lower sample mean. Consequently, we would expect these two things to be positively correlated.

Now, using the fact that the covariance operator is linear in each of its arguments, you get:

$$\begin{align} \mathbb{C}(X_1, \bar{X}_n) &= \mathbb{C} \Bigg( X_1, \frac{1}{n} \sum_{i=1}^n X_i \Bigg) \\[6pt] &= \frac{1}{n} \sum_{i=1}^n \mathbb{C} ( X_1, X_i ) \\[6pt] &= \frac{1}{n} \mathbb{C}(X_1, X_1) \\[6pt] &= \frac{1}{n} \mathbb{V}(X_1) \\[6pt] &= \frac{\sigma^2}{n}. \\[6pt] \end{align}$$

This confirms that the covariance is positive so long as $\sigma>0$. As $n \rightarrow \infty$ the sample mean depends less and less on the value $X_1$, and the covariance diminishes to zero. (Note also that this result does not depend on the assumption of normality of the underlying variables; it holds for any underlying distribution with finite variance.)

$\endgroup$
3
  • 1
    $\begingroup$ (+1). I think I'm having an intellectual blackout: How do you get from the second line of the equation to the third? I'm suspecting because of the IID assumption? $\endgroup$ Apr 14, 2021 at 15:49
  • 1
    $\begingroup$ @COOLSerdash: Since the data values are IID they are uncorrelated, so you have $\mathbb{C}(X_1, X_i) = 0$ for all $i \neq 1$. Consequently, the only non-zero term in the sum is $\mathbb{C}(X_1, X_1)$. $\endgroup$
    – Ben
    Apr 15, 2021 at 1:55
  • $\begingroup$ Thanks Ben, I suspected it was something easy ... $\endgroup$ Apr 15, 2021 at 9:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.