# Expected vallue calculation of i.i.d. random variables

Suppose $$X_1,X_2,\ldots,X_n$$ are a sequence of i.i.d. random variables with mean $$\mu$$ and variance $$\sigma^2$$. Define the sample mean $$\bar{X} := \frac{1}{n} \sum_{i=1}^{n} X_i$$, which we know is an unbiased estimator of the sample mean with mean $$\mu$$ and variance $$\sigma^2/n$$, i.e.

\begin{align*} \mathbb{E}[\bar{X}] &= \mu, \\ \textrm{Var}(\bar{X}) := \mathbb{E}[(\bar{X} - \mu)^2] &= \frac{\sigma^2}{n}. \end{align*}

I am interested in calculating the expected value of the quantity $$Z_n := \sum_{i=1}^{n} (X_i - \bar{X})^2$$, but my results don't make sense. First, I expand the expectation to get

\begin{align*} \mathbb{E}[Z_n] &= \mathbb{E}\bigg[\sum_{i=1}^{n}(X_i - \bar{X})^2\bigg] = \mathbb{E}\bigg[(X_1 - \bar{X})^2 + \ldots + (X_n - \bar{X})^2\bigg] \\ &= \sum_{i=1}^{n} \mathbb{E}[(X_i - \bar{X})^2] = \sum_{i=1}^{n} \mathbb{E}[X_i^2 + \bar{X}^2 - 2X_i\bar{X}] \\ &= \sum_{i=1}^{n}(\mathbb{E}[X_i^2] + \mathbb{E}[\bar{X}^2] - 2\mathbb{E}[X_i\bar{X}]). \end{align*}

Thus, there are three expectations to compute. First, since each $$X_i$$ is i.i.d, it follows from the definition of variance that $$\sigma^2 = \mathbb{E}[X_i^2] - \mathbb{E}[X_i]^2 \Rightarrow \mathbb{E}[X_i^2] = \sigma^2 + \mu^2$$. Additionally, the same argument applies to the expected value of the squared sample mean, i.e., $$\mathbb{E}[\bar{X}^2] = \sigma^2/n + \mu^2$$.

The last expectation, $$\mathbb{E}[X_i,\bar{X}]$$ is a bit more tricky to compute. First, let us plug in what we currently have, which gives

$$\mathbb{E}[Z_n] = \sum_{i=1}^{n} \bigg[(\sigma^2 + \mu^2) + \bigg(\frac{\sigma^2}{n} + \mu^2\bigg) -2\mathbb{E}[X_i\bar{X}]\bigg] = 2\mu^2n + (n+1)\sigma^2 - 2\sum_{i=1}^{n}\mathbb{E}[X_i\bar{X}].$$

Now, for the last term, let us use the definition of the sample mean to get

$$\sum_{i=1}^{n} \mathbb{E}[X_i\bar{X}] = \sum_{i=1}^{n} \mathbb{E}\bigg[X_i\bigg(\frac{1}{n}\sum_{j=1}^{n}X_j\bigg)\bigg] = \frac{1}{n}\sum_{i=1}^{n}\sum_{j=1}^{n}\mathbb{E}[X_iX_j],$$ where I used the linearity of the expectation in the last equality. Noting that $$\textrm{Cov}(X_i,X_j) = 0$$ for all $$i \neq j$$ since each $$X_i$$ are independent, we see that $$\textrm{Cov}(X_i,X_j) = \mathbb{E}[X_iX_j] - \mu^2 = 0$$ for all $$i \neq j$$, which implies $$\mathbb{E}[X_iX_j] = \mu^2$$ for all $$i \neq j$$. Similarly, for all $$i = j$$, we have $$\textrm{Cov}(X_i,X_j) = \textrm{Cov}(X_i,X_i) = \sigma^2$$, by definition. Thus, if we break up that double sum into a double sum when $$i = j$$ and a double sum when $$i \neq j$$, we get

$$\sum_{i=1}^{n}\mathbb{E}[X_i\bar{X}] = \frac{1}{n}(n\mu^2 + n\sigma^2) = \mu^2 + \sigma^2.$$

Plugging this back in gives

$$\mathbb{E}[Z_n] = 2\mu^2n + (n + 1)\sigma^2 - 2(\sigma^2 + \mu^2) = \boxed{ (n-1)(2\mu^2 + \sigma^2) }$$

My question is what is the physical significance of this $$Z_n$$ that I'm trying to calculate, and is the calculation correct?

Let's start with: \begin{align*} Z_n&=\sum_{i=1}^n(X_i-\bar{X})^2=\sum_{i=1}^n(X_i^2-2X_i\bar{X}+\bar{X}^2)\\ &=\sum_{i=1}^nX_i^2-2\left(\sum_{i=1}^nX_i\right)\bar{X}+n\bar{X}^2\\ &=\sum_{i=1}^nX_i^2-2n\bar{X}\bar{X}+n\bar{X}^2=\sum_{i=1}^nX_i^2-n\bar{X}^2 \end{align*} Then $$E[Z_n]=E\left[\sum_{i=1}^nX_i^2\right]-nE[\bar{X}^2]\overset{\mathrm{iid}}{=}nE[X^2]-nE[\bar{X}^2]$$ where $$X\sim X_i$$, $$i=1,\dots,n$$. Since $$\sigma^2=E[X^2]-\mu^2$$ and $$[\bar{X}^2]=\frac{\sigma^2}{n}+\mu^2$$, $$E[Z_n]=n\left(\sigma^2+\mu^2-\frac{\sigma^2}{n}+\mu^2\right)=(n-1)\sigma^2$$ But what is $$Z_n$$? $$Z_n$$ is just $$Z_n=n\hat\sigma^2_n=(n-1)S^2_n$$ where $$\hat\sigma^2_n=\frac1n\sum_{i=1}^n(X_i-\bar{X})^2$$ is the sample variance and $$S^2_n=\frac{1}{n-1}\sum_{i=1}^n(X_i-\bar{X})^2$$ is the unbiased sample variance: $$E[\hat\sigma^2_n]=\frac{n-1}{n}\sigma^2,\qquad E[S^2_n]=\sigma^2$$