# Covariance between partitions of a normal distribution

A bit of a contrived example, but if I had a sample of $$X_1,\dots,X_n \stackrel{iid}{\sim} N(\mu,\sigma^2)$$ (in this case $$\mu$$ is unknown but $$\sigma^2$$ is known), and then calculated the arithmetic mean of the sample, how do I find the covariance between $$Cov(X_1,\overline{X})$$

I know (at least I think I do) that it should become equal to Var$$(\overline{X})$$, which I can state as $$\frac{\sigma^2}{n}$$ because $$\sigma^2$$ is known.

I rewrite as:

$$Cov(X_1,\overline{X})=E(X_1\overline{X})-E(X_1)E(\overline{X})$$

which becomes $$E(X_1\overline{X})-E(\overline{X})^2$$ but evaluating that first part is causing me some issues. Am I on the right track?

Thank you.

• The answer below shows that the covariance does not depend on the normality assumption. If you were to use the normality assumption, then $X_1-\overline X$ and $\overline X$ are independent by Basu's theorem. So $\operatorname{Cov}(X_1-\overline X,\overline X)=0\implies \operatorname{Cov}(X_1,\overline X)=\operatorname{Var}(\overline X)$. Sep 9 '21 at 12:18

You can write

$$X_1 \bar{X} = \frac{1}{n}\sum_{i=1}^n X_1 X_i$$

If you take the expectation you get

$$\frac{1}{n}\sum_{i=1}^n \mathbb E(X_1 X_i) = \frac{1}{n}\left( \mathbb E(X_1^2) + \sum_{i=2}^n\mathbb E( X_1 X_i)\right)$$ Since (I assume) $$X_i \perp \!\!\! \perp X_1$$ for $$i \neq 1$$ then $$\mathbb E( X_1 X_i) = \mathbb E(X_1) \mathbb E(X_i) = \mu^2$$

Thus,

\begin{align*} \frac{1}{n}\sum_{i=1}^n \mathbb E(X_1 X_i) &= \frac{1}{n}\left (\mathbb E(X_1^2) + (n-1) \mu^2 \right ) \\ &= \frac{1}{n}\left( \sigma^2 + \mu^2 + (n-1)\mu^2 \right) \\ &= \frac{\sigma^2}{n} + \mu^2 \end{align*}

And finally,

\begin{align*} \text{Cov}(X_1,\bar{X}) &= \frac{\sigma^2}{n} + \mu^2 - \mathbb E(X_1)\mathbb E(\bar{X}) \\ &= \frac{\sigma^2}{n} \end{align*}

Another possibility to compute $$\text{Cov}(X_1,\bar{X})$$ is to use the bilinearity of the covariance, i.e for random variables $$(X_1,\dots,X_k,Y_1,\dots,Y_n)$$ and constants $$(\alpha_1,\dots,\alpha_k,\beta_1,\dots,\beta_n)$$ we have

$$\text{Cov} \left( \sum_{i=1}^k \alpha_1 X_i, \sum_{j=1}^n \beta_j Y_j \right ) = \sum_{i=1}^k \sum_{j=1}^n \alpha_i \beta_j X_i Y_j$$

$$\text{Cov}(X_1,\bar{X}) = \frac{1}{n} \sum_{i=1}^n \text{Cov}(X_1,X_i)$$
By independance of the vector $$(X_1,\dots,X_n)$$, $$\text{Cov}(X_1,X_i) = 0$$ for $$i \neq 1$$ thus
\begin{align*} \text{Cov}(X_1,\bar{X}) &= \frac{1}{n} \text{Cov}(X_1,X_1) \\ &= \frac{1}{n} \text{Var}(X_1) \\ &= \frac{\sigma^2}{n} \end{align*}