I'm trying to reconstruct variational autoencoder situation with a more "imaginable" example, known all properties. I was playing around, and can't find a way how can for example the sum of two normal distributions ($P(Z|X_1)$, $P(Z|X_2)$) with different means, and standard deviations multiplied with $P(X)$ can form $P(Z)$ - a normal distributon.
Just for example we're checking a group's IQ, and region. The group consists of 10000 participants.
$X$ states regions.
- $X_1$, region one has a probability of 0.2, so 2000 participants
- $X_2$, region two has a probability of 0.8, so 8000 participants
$Z$ states the IQ. It is a normal distribution with mean 100, and standard deviation 10.
Out of both regions, participants IQ is:
IQ P(Z[IQ]) quantity 60-70: 0.0020 - 20 70-80: 0.0210 - 210 80-90: 0.1360 - 1360 90-100: 0.3410 - 3410 100-110: 0.3410 - 3410 110-120: 0.1360 - 1360 120-130: 0.0210 - 210 130-140: 0.0020 - 20
The only way I got the same results back if $P(Z|X_i) = P(Z)$:
X: 4, 42, 272, 682, 682, 272, 42, 4 X: 16, 168, 1088, 2728, 2728, 1088, 168, 16
I was trying out different setups for $P(Z|X_i)$ not be equal to $P(Z)$, with different means, or standard deviations, but by simply looking at the graphs, it is clear that the shape of them regardless of $P(X)$ won't form a normal distribution. According to my tries, I can't see a way to form a gaussian from two gaussians that aren't equal. However, in VAEs we're trying to accomplish that, so there must be something I'm getting wrong.