If $P(Z)$ is normal distribution, can $P(Z|X_i)$ be normal distribution with different mean, or standard deviation?

Question

I'm trying to reconstruct variational autoencoder situation with a more "imaginable" example, known all properties. I was playing around, and can't find a way how can for example the sum of two normal distributions ($P(Z|X_1)$, $P(Z|X_2)$) with different means, and standard deviations multiplied with $P(X)$ can form $P(Z)$ - a normal distributon.

Just for example we're checking a group's IQ, and region. The group consists of 10000 participants.

$X$ states regions.

• $X_1$, region one has a probability of 0.2, so 2000 participants
• $X_2$, region two has a probability of 0.8, so 8000 participants

$Z$ states the IQ. It is a normal distribution with mean 100, and standard deviation 10.

Out of both regions, participants IQ is:

    IQ       P(Z[IQ]) quantity
    60-70:   0.0020 - 20
    70-80:   0.0210 - 210
    80-90:   0.1360 - 1360
    90-100:  0.3410 - 3410
    100-110: 0.3410 - 3410
    110-120: 0.1360 - 1360
    120-130: 0.0210 - 210
    130-140: 0.0020 - 20

The only way I got the same results back if $P(Z|X_i) = P(Z)$:

  X[1]: 4,  42,  272,  682,  682,  272,  42,  4
  X[2]: 16, 168, 1088, 2728, 2728, 1088, 168, 16

I was trying out different setups for $P(Z|X_i)$ not be equal to $P(Z)$, with different means, or standard deviations, but by simply looking at the graphs, it is clear that the shape of them regardless of $P(X)$ won't form a normal distribution. According to my tries, I can't see a way to form a gaussian from two gaussians that aren't equal. However, in VAEs we're trying to accomplish that, so there must be something I'm getting wrong.

Answer 1

A mixture of Gaussians$$p_1\mathcal N(\mu_1,\sigma^2_1)+(1-p_1)\mathcal N(\mu_2,\sigma^2_2)$$is only a Gaussian when $$\mu_1=\mu_2\,\quad \sigma_1=\sigma_2$$ or when $$p_1\in\{0,1\}$$