Connection between sum of normally distributed random variables and mixture of normal distributions

Question

If you have two independent random variables that are normally distributed (not necessarily jointly so), then their sum is also normally distributed, which e.g. means that its excess kurtosis is $0$.

On the other hand in case the mixture of one-dimensional normal distributions the mixture distribution can display non-trivial higher-order moments such as skewness and kurtosis (fat tails) and multi-modality, even in the absence of such features within the components themselves (see also this video for an easy enough example). This means that the result doesn't have to be normally distributed.

My question
How can you reconcile these two results and what is their connection?

Answer 1

It's important to make the distinction between a sum of normal random variables and a mixture of normal random variables.

As an example, consider independent random variables $X_1\sim N(\mu_1,\sigma_1^2)$, $X_2\sim N(\mu_2,\sigma_2^2)$, $\alpha_1\in\left[0,1\right]$, and $\alpha_2=1-\alpha_1$.

Let $Y=X_1+X_2$. $Y$ is the sum of two independent normal random variables. What's the probability that $Y$ is less than or equal to zero, $P(Y\leq0)$? It's simply the probability that a $N(\mu_1+\mu_2,\sigma_1^2+\sigma_2^2)$ random variable is less than or equal to zero because the sum of two independent normal random variables is another normal random variable whose mean is the sum of the means and whose variance is the sum of the variances.

Let $Z$ be a mixture of $X_1$ and $X_2$ with respective weights $\alpha_1$ and $\alpha_2$. Notice that $Z\neq \alpha_1X_1+\alpha_2X_2$. The fact that $Z$ is defined as a mixture with those specific weights means that the CDF of $Z$ is $F_Z(z)=\alpha_1F_1(z)+\alpha_2F_2(z)$, where $F_1$ and $F_2$ are the CDFs of $X_1$ and $X_2$, respectively. So what is the probability that $Z$ is less than or equal to zero, $P(Z\leq0)$? It's $F_Z(0)=\alpha_1F_1(0)+\alpha_2F_2(0)$.

Answer 2

Max has correctly pointed out the difference between sums of independent normals and mixtures of normals. However, he did not show directly how that answers your question.

Simply stated the two distributions are vastly different. The sum is normally distributed. A mixture of two or three normals is not. In fact, it will be bimodal when two distributions are included with very different means and nearly equal weights. When one distribution has a high weight and the other is far to the right with a small weight this will induce skewness and possibly kurtosis that is larger than for a normal distribution. In the case of three distributions with the middle having a weight of 0.8 and the other two equally shifted one to the right and the other to the left with a weight of 0.1, each the distribution will be symmetric with heavy tails. Plot some of the densities and the non-normality will be obvious.