Variance of the average of independent normal observations with different means and variance

Question

I am a little confused here on finding a probability but I can figure out the end I just need help on the intermediate steps.

So we have 55 independent normal observations, which all have the same mean $\mu$. The first 50 observations have variance $\sigma_1^2$ and the last 5 observations have variance $\sigma_2^2$.

What is the variance of the average of all 55 observations? I left out some of the numbers here so I can scrutinize it theoretically.

I've tried a few things.

• The sum of both variances. I ruled this out because one of the subsets has a much higher variance than the other, much smaller subset. I felt like this would somehow influence the ending variance/standard deviation, in the way that outliers do.
• The sum of both variances, weighted by the proportion of the entire set they make up. That is to say, if one of the subsets is 10/11ths of the data and the other is 1/11th, then their respective variances would take up that percentage of the variance of the average of all 55.
• The sum of both variances, divided by 55, the total number of all observations.
• The sum of both variances divided by $55^2$.

What about the average, while I'm asking? Do I add them together? Do divide $\mu$ by 50 and 5 respectively, and then add them? Do I add $\mu + \mu$ and then divide by 55? Is the average simply $\mu$?

Any help would be appreciated.

Answer 1

The variance of the sum of two independent Gaussian random variables is just the sum of their variances: $$ \operatorname{Var}\left( X_1 + X_2 \right) = \operatorname{Var}\left( X_1 \right) + \operatorname{Var}\left( X_2 \right) $$

This can be extended to an arbitrary number $N$ of such random variables, because the sum of two Gaussians is itself Gaussian:

1. Let $S_n = \sum_{k=1}^n X_k$ where every $X_k$ is Gaussian
2. $S_1 = X_1$ and $S_2 = X_1 + X_2$ are Gaussian
3. If $S_{n-1}$ is Gaussian, then $S_n = S_{n-1} + X_n$ is Gaussian
4. By induction, $S_n$ is Gaussian for any $n$
5. Let $V_n = \operatorname{Var}\left( S_n \right)$
6. $V_1 = \operatorname{Var}\left( X_1 \right)$ and $V_2 = \operatorname{Var}\left( X_1 \right) + \operatorname{Var}\left( X_1 \right)$
7. Since $S_{n-1}$ is Gaussian, $V_n = V_{n-1} + \operatorname{Var}\left( X_n \right)$
8. $$\begin{align} V_n & = \operatorname{Var}\left( \sum_{k=1}^n X_k \right) \\ & = \operatorname{Var}\left( S_{n} \right) \\ & = \operatorname{Var}\left( S_{n-1} \right) + \operatorname{Var}\left( X_n \right) \\ & = \operatorname{Var}\left( S_{n-2} \right) + \operatorname{Var}\left( X_{n-1} \right) + \operatorname{Var}\left( X_n \right) \\ & \cdots \\ & \operatorname{Var}\left( X_1 \right) + \dots + \operatorname{Var}\left( X_n \right) \end{align}$$

Or, more compactly,

$$ \operatorname{Var}\left(\sum_{i=1}^N X_i\right) = \sum_{i=1}^N \operatorname{Var}\left(X_i\right) $$

Now let $\bar X$ denote the average, $\bar X = \frac{1}{N} \sum_{i=1}^N X_i$. It is a basic property of the variance operator that $\operatorname{Var}\left(aY\right) = a^2 \operatorname{Var}\left(Y\right)$ for any random variable $Y$. Then you have your answer: $$ \operatorname{Var}\left( \bar X \right) = \operatorname{Var}\left(\frac{1}{N} \sum_{i=1}^N X_i\right) = \frac{1}{N^2} \operatorname{Var}\left( \sum_{i=1}^N X_i\right) = \frac{1}{N^2} \sum_{i=1}^N \operatorname{Var}\left(X_i\right) $$

So for the numbers you stated, you would calculate: $$ \frac{1}{55^2} \left( 50 \sigma_1^2 + 5\sigma_2^2 \right) $$