Mixing Distributions with Certain Probability? Suppose I have two distributions $X \sim N(a, \sigma ^2)$ and $Y \sim N(b,\sigma^2)$. From these distributions, I create a third distribution $Z$ so that there is a $\frac 12$ chance that $Z$ comes from $X$ and a $\frac 12$ chance that $Z$ comes from $Y$. Question: What is the distribution of $Z$ (i.e. expectation, variance).
It seems intuitive that $Z=\frac 12 X + \frac 12 Y$ Here's my reasoning, which I think is faulty (as I will explain later). The PDF for $Z$ is 
$P(Z\le z)= P(Z \le z | Z \text{ chosen from } X) * P(Z \text{ chosen from } X) +P(Z \le z | Z \text{ chosen from } Y) * P(Z \text{ chosen from } Y)$
$=\frac 12 \text{PDF for } X + \frac 12 \text {PDF for } Y$.
So $Z\sim \frac 12 X + \frac 12 Y$.
Which is normal, and one can calculate the mean and the variance easily to find that $Z \sim N(\frac 12 a + \frac 12 b, \frac 12 \sigma ^2)$
But what- what happened here? Specifically, what if I let $a=b$. Then $Z \sim N(a,\frac 12 \sigma ^2)$, but $Z$ should clearly be just $N(a,\sigma ^2)$. Where did I go wrong, and what should be the correct answer?
 A: I think there are a couple issues here.  One you introduce the density functions for $X$ and $Y$ incorrectly when calculating the distribution function of $Z$.  You should have gotten
$$
P(Z \leq z) = \frac{1}{2} \Phi_{a, \sigma^2}(z) + \frac{1}{2} \Phi_{b, \sigma^2}(z) 
$$
where $\Phi_{\mu, \sigma^2}(z)$ is the distribution function for a normal$(\mu, \sigma^2)$.  This does not mean that $Z$ is a sum of $X$ and $Y$.  Notice that even if somehow $Z$ were a linear combination of $X$ and $Y$ you would still need to know their joint distribution to determine the distribution of $Z$, something which you never specify.  This tells you that there's a gap in your reasoning.
$Z$ instead follows a mixture normal distribution which is most certainly not normal (in general it has more than one mode, for instance).  If you want to calculate the mean and variance, say, then you can try conditioning on the distribution from which we're drawing (that of $X$ or $Y$) and take an expectation as you do when finding $P(Z \leq z)$.
Edit based on comments
To calculate the expectation and variance of $Z$ let's call the mixing indicator $I$ and have $I = 1$ if we sample from the distribution of $X$ and $I = 0$ if we sample from the distribution of $Y$.  This means that $Z = IX + (1 - I)Y$.  To get both the mean and variance we condition on $I$ (we'll denote the mixing probability with $p$ just to make this more general):
\begin{align}
\text{E}(Z) &= \text{E}[\text{E}(Z \mid I)] \\
&= \text{E}[I a + (1 - I) b] \\
&= p a + (1 - p) b .
\end{align}
The variance however is a little bit more complicated due to the formula
\begin{align}
\text{Var}(Z) &= \text{Var}[ \text{E}(Z \mid I)] + \text{E}[ \text{Var}(Z \mid I)].
\end{align}
Note that we cannot simply average variances as we do with expectations.  One quick counterexample to prove this would be the Bernoulli distribution itself, viewed as a mixture of a point mass at one and a point mass at zero.  Each of these random variables is just a constant and so has zero variance, but their mixture has positive variance in general.  In any event, for the first term in the above formula we have
\begin{align}
\text{Var}[ \text{E}(Z \mid I)] &= \text{Var}(I a + (1 - I) b) \\
&= a^2 \text{Var}(I) + b^2 \text{Var}(1 - I) + 2 a b \text{Cov}(I, 1 - I) \\
&= a^2 p (1 - p) + b^2 p (1 - p) - 2abp (1 - p) \\
&= p(1 -p) (a - b)^2.
\end{align}
And for the second
\begin{align}
\text{E}[ \text{Var}(Z \mid I)] &= \text{E} [ \text{Var}(IX + (1 - I) Y \mid I) ] \\
&= \text{E} [ I^2 \sigma_x^2 + (1 - I)^2 \sigma_y^2 + 2 I(1 - I) \sigma_{xy} ] \\
&= \text{E} [ I \sigma_x^2 + (1 - I) \sigma_y^2] \\
&= p \sigma_x^2 + (1 - p) \sigma_y^2 .
\end{align}
Putting everything together we get
\begin{align}
\text{Var}(Z) &= p(1 - p) (a - b)^2 + p \sigma_x^2 + (1 - p) \sigma_y^2 .
\end{align}
