Connection between sum of normally distributed random variables and mixture of normal distributions

If you have two independent random variables that are normally distributed (not necessarily jointly so), then their sum is also normally distributed, which e.g. means that its excess kurtosis is $0$.

On the other hand in case the mixture of one-dimensional normal distributions the mixture distribution can display non-trivial higher-order moments such as skewness and kurtosis (fat tails) and multi-modality, even in the absence of such features within the components themselves (see also this video for an easy enough example). This means that the result doesn't have to be normally distributed.

My question
How can you reconcile these two results and what is their connection?

• A mixture of two normals is not the sum of two normal random variables. It's a random variable whose PDF and CDF are weighted sums of the individuals PDFs and CDFs. – assumednormal Jul 29 '12 at 16:43
• @Max, that's an answer! – Dilip Sarwate Jul 29 '12 at 16:49
• I'll write it up now. – assumednormal Jul 29 '12 at 17:05
• If you have suspicions that something might not be true, it's good to look for simple (counter)examples. Consider $X_1$ and $X_2$ independent Bernoulli random variables. A mixture of these is still Bernoulli (why?), but the sum of them could, in general, take the value $2$ (in addition to $0$ or $1$). So, clearly they can't be the same thing. :) – cardinal Jul 29 '12 at 21:56
• You don't have to go to higher moments to see that a not all mixtures are normal. Mixtures of normal distributions don't have to be unimodal, although they can be. In addition, since distinct normal distributions have distinct tail asymptotics, you can read off the components from the tails. – Douglas Zare Jul 30 '12 at 4:48

As an example, consider independent random variables $X_1\sim N(\mu_1,\sigma_1^2)$, $X_2\sim N(\mu_2,\sigma_2^2)$, $\alpha_1\in\left[0,1\right]$, and $\alpha_2=1-\alpha_1$.
Let $Y=X_1+X_2$. $Y$ is the sum of two independent normal random variables. What's the probability that $Y$ is less than or equal to zero, $P(Y\leq0)$? It's simply the probability that a $N(\mu_1+\mu_2,\sigma_1^2+\sigma_2^2)$ random variable is less than or equal to zero because the sum of two independent normal random variables is another normal random variable whose mean is the sum of the means and whose variance is the sum of the variances.
Let $Z$ be a mixture of $X_1$ and $X_2$ with respective weights $\alpha_1$ and $\alpha_2$. Notice that $Z\neq \alpha_1X_1+\alpha_2X_2$. The fact that $Z$ is defined as a mixture with those specific weights means that the CDF of $Z$ is $F_Z(z)=\alpha_1F_1(z)+\alpha_2F_2(z)$, where $F_1$ and $F_2$ are the CDFs of $X_1$ and $X_2$, respectively. So what is the probability that $Z$ is less than or equal to zero, $P(Z\leq0)$? It's $F_Z(0)=\alpha_1F_1(0)+\alpha_2F_2(0)$.