1
$\begingroup$

A bit of a contrived example, but if I had a sample of $X_1,\dots,X_n \stackrel{iid}{\sim} N(\mu,\sigma^2)$ (in this case $\mu$ is unknown but $\sigma^2$ is known), and then calculated the arithmetic mean of the sample, how do I find the covariance between $$Cov(X_1,\overline{X})$$

I know (at least I think I do) that it should become equal to Var$(\overline{X})$, which I can state as $\frac{\sigma^2}{n}$ because $\sigma^2$ is known.

I rewrite as:

$$Cov(X_1,\overline{X})=E(X_1\overline{X})-E(X_1)E(\overline{X})$$

which becomes $E(X_1\overline{X})-E(\overline{X})^2$ but evaluating that first part is causing me some issues. Am I on the right track?

Thank you.

$\endgroup$
1
  • 2
    $\begingroup$ The answer below shows that the covariance does not depend on the normality assumption. If you were to use the normality assumption, then $X_1-\overline X$ and $\overline X$ are independent by Basu's theorem. So $\operatorname{Cov}(X_1-\overline X,\overline X)=0\implies \operatorname{Cov}(X_1,\overline X)=\operatorname{Var}(\overline X)$. $\endgroup$ Sep 9 '21 at 12:18
1
$\begingroup$

You can write

$$ X_1 \bar{X} = \frac{1}{n}\sum_{i=1}^n X_1 X_i $$

If you take the expectation you get

$$ \frac{1}{n}\sum_{i=1}^n \mathbb E(X_1 X_i) = \frac{1}{n}\left( \mathbb E(X_1^2) + \sum_{i=2}^n\mathbb E( X_1 X_i)\right) $$ Since (I assume) $X_i \perp \!\!\! \perp X_1$ for $i \neq 1$ then $$ \mathbb E( X_1 X_i) = \mathbb E(X_1) \mathbb E(X_i) = \mu^2 $$

Thus,

\begin{align*} \frac{1}{n}\sum_{i=1}^n \mathbb E(X_1 X_i) &= \frac{1}{n}\left (\mathbb E(X_1^2) + (n-1) \mu^2 \right ) \\ &= \frac{1}{n}\left( \sigma^2 + \mu^2 + (n-1)\mu^2 \right) \\ &= \frac{\sigma^2}{n} + \mu^2 \end{align*}

And finally,

\begin{align*} \text{Cov}(X_1,\bar{X}) &= \frac{\sigma^2}{n} + \mu^2 - \mathbb E(X_1)\mathbb E(\bar{X}) \\ &= \frac{\sigma^2}{n} \end{align*}


Another possibility to compute $\text{Cov}(X_1,\bar{X})$ is to use the bilinearity of the covariance, i.e for random variables $(X_1,\dots,X_k,Y_1,\dots,Y_n)$ and constants $(\alpha_1,\dots,\alpha_k,\beta_1,\dots,\beta_n)$ we have

$$ \text{Cov} \left( \sum_{i=1}^k \alpha_1 X_i, \sum_{j=1}^n \beta_j Y_j \right ) = \sum_{i=1}^k \sum_{j=1}^n \alpha_i \beta_j X_i Y_j $$

In your case it yields,

$$ \text{Cov}(X_1,\bar{X}) = \frac{1}{n} \sum_{i=1}^n \text{Cov}(X_1,X_i) $$

By independance of the vector $(X_1,\dots,X_n)$, $\text{Cov}(X_1,X_i) = 0$ for $i \neq 1$ thus

\begin{align*} \text{Cov}(X_1,\bar{X}) &= \frac{1}{n} \text{Cov}(X_1,X_1) \\ &= \frac{1}{n} \text{Var}(X_1) \\ &= \frac{\sigma^2}{n} \end{align*}

$\endgroup$
2
  • $\begingroup$ Thank you @periwinkle, this makes perfect sense now! I will update my question to state that the observations are independent as you correctly assumed. $\endgroup$
    – TNoms
    Sep 9 '21 at 8:39
  • $\begingroup$ The last approach, beginning at "By independence," is so straightforward and simple that it ought to lead off this answer. $\endgroup$
    – whuber
    Sep 9 '21 at 12:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.