Covariance between partitions of a normal distribution

Question

A bit of a contrived example, but if I had a sample of $X_1,\dots,X_n \stackrel{iid}{\sim} N(\mu,\sigma^2)$ (in this case $\mu$ is unknown but $\sigma^2$ is known), and then calculated the arithmetic mean of the sample, how do I find the covariance between $$Cov(X_1,\overline{X})$$

I know (at least I think I do) that it should become equal to Var$(\overline{X})$, which I can state as $\frac{\sigma^2}{n}$ because $\sigma^2$ is known.

I rewrite as:

$$Cov(X_1,\overline{X})=E(X_1\overline{X})-E(X_1)E(\overline{X})$$

which becomes $E(X_1\overline{X})-E(\overline{X})^2$ but evaluating that first part is causing me some issues. Am I on the right track?

Thank you.

Answer 1

You can write

$$ X_1 \bar{X} = \frac{1}{n}\sum_{i=1}^n X_1 X_i $$

If you take the expectation you get

$$ \frac{1}{n}\sum_{i=1}^n \mathbb E(X_1 X_i) = \frac{1}{n}\left( \mathbb E(X_1^2) + \sum_{i=2}^n\mathbb E( X_1 X_i)\right) $$ Since (I assume) $X_i \perp \!\!\! \perp X_1$ for $i \neq 1$ then $$ \mathbb E( X_1 X_i) = \mathbb E(X_1) \mathbb E(X_i) = \mu^2 $$

Thus,

\begin{align*} \frac{1}{n}\sum_{i=1}^n \mathbb E(X_1 X_i) &= \frac{1}{n}\left (\mathbb E(X_1^2) + (n-1) \mu^2 \right ) \\ &= \frac{1}{n}\left( \sigma^2 + \mu^2 + (n-1)\mu^2 \right) \\ &= \frac{\sigma^2}{n} + \mu^2 \end{align*}

And finally,

\begin{align*} \text{Cov}(X_1,\bar{X}) &= \frac{\sigma^2}{n} + \mu^2 - \mathbb E(X_1)\mathbb E(\bar{X}) \\ &= \frac{\sigma^2}{n} \end{align*}

---

Another possibility to compute $\text{Cov}(X_1,\bar{X})$ is to use the bilinearity of the covariance, i.e for random variables $(X_1,\dots,X_k,Y_1,\dots,Y_n)$ and constants $(\alpha_1,\dots,\alpha_k,\beta_1,\dots,\beta_n)$ we have

$$ \text{Cov} \left( \sum_{i=1}^k \alpha_1 X_i, \sum_{j=1}^n \beta_j Y_j \right ) = \sum_{i=1}^k \sum_{j=1}^n \alpha_i \beta_j X_i Y_j $$

In your case it yields,

$$ \text{Cov}(X_1,\bar{X}) = \frac{1}{n} \sum_{i=1}^n \text{Cov}(X_1,X_i) $$

By independance of the vector $(X_1,\dots,X_n)$, $\text{Cov}(X_1,X_i) = 0$ for $i \neq 1$ thus

\begin{align*} \text{Cov}(X_1,\bar{X}) &= \frac{1}{n} \text{Cov}(X_1,X_1) \\ &= \frac{1}{n} \text{Var}(X_1) \\ &= \frac{\sigma^2}{n} \end{align*}