Connection between sum of normally distributed random variables and mixture of normal distributions

If you have two independent random variables that are normally distributed (not necessarily jointly so), then their sum is also normally distributed, which e.g. means that its excess kurtosis is $0$.

On the other hand in case the mixture of one-dimensional normal distributions the mixture distribution can display non-trivial higher-order moments such as skewness and kurtosis (fat tails) and multi-modality, even in the absence of such features within the components themselves (see also this video for an easy enough example). This means that the result doesn't have to be normally distributed.

My question
How can you reconcile these two results and what is their connection?

• A mixture of two normals is not the sum of two normal random variables. It's a random variable whose PDF and CDF are weighted sums of the individuals PDFs and CDFs. Commented Jul 29, 2012 at 16:43
• @Max, that's an answer! Commented Jul 29, 2012 at 16:49
• I'll write it up now. Commented Jul 29, 2012 at 17:05
• If you have suspicions that something might not be true, it's good to look for simple (counter)examples. Consider $X_1$ and $X_2$ independent Bernoulli random variables. A mixture of these is still Bernoulli (why?), but the sum of them could, in general, take the value $2$ (in addition to $0$ or $1$). So, clearly they can't be the same thing. :) Commented Jul 29, 2012 at 21:56
• You don't have to go to higher moments to see that a not all mixtures are normal. Mixtures of normal distributions don't have to be unimodal, although they can be. In addition, since distinct normal distributions have distinct tail asymptotics, you can read off the components from the tails. Commented Jul 30, 2012 at 4:48

2 Answers

It's important to make the distinction between a sum of normal random variables and a mixture of normal random variables.

As an example, consider independent random variables $X_1\sim N(\mu_1,\sigma_1^2)$, $X_2\sim N(\mu_2,\sigma_2^2)$, $\alpha_1\in\left[0,1\right]$, and $\alpha_2=1-\alpha_1$.

Let $Y=X_1+X_2$. $Y$ is the sum of two independent normal random variables. What's the probability that $Y$ is less than or equal to zero, $P(Y\leq0)$? It's simply the probability that a $N(\mu_1+\mu_2,\sigma_1^2+\sigma_2^2)$ random variable is less than or equal to zero because the sum of two independent normal random variables is another normal random variable whose mean is the sum of the means and whose variance is the sum of the variances.

Let $Z$ be a mixture of $X_1$ and $X_2$ with respective weights $\alpha_1$ and $\alpha_2$. Notice that $Z\neq \alpha_1X_1+\alpha_2X_2$. The fact that $Z$ is defined as a mixture with those specific weights means that the CDF of $Z$ is $F_Z(z)=\alpha_1F_1(z)+\alpha_2F_2(z)$, where $F_1$ and $F_2$ are the CDFs of $X_1$ and $X_2$, respectively. So what is the probability that $Z$ is less than or equal to zero, $P(Z\leq0)$? It's $F_Z(0)=\alpha_1F_1(0)+\alpha_2F_2(0)$.

Max has correctly pointed out the difference between sums of independent normals and mixtures of normals. However, he did not show directly how that answers your question.

Simply stated the two distributions are vastly different. The sum is normally distributed. A mixture of two or three normals is not. In fact, it will be bimodal when two distributions are included with very different means and nearly equal weights. When one distribution has a high weight and the other is far to the right with a small weight this will induce skewness and possibly kurtosis that is larger than for a normal distribution. In the case of three distributions with the middle having a weight of 0.8 and the other two equally shifted one to the right and the other to the left with a weight of 0.1, each the distribution will be symmetric with heavy tails. Plot some of the densities and the non-normality will be obvious.