Question about sums of Gasussian Mixture models

Question

This question is strongly based on the result given in HERE

Due to some research I am currently conducting, I've found myself in a situation, where I deal with mixtures of gaussian densities, called Gaussian Mixture Models, or GMM for short (mixtures of gaussian densities, with the constraint that weights in a mixture sum to 1). At some point, I've begun to wonder whether or not a sum of GMM's will still be a GMM. Then I've stumbled upon the above reasoning. Based on that we can see that

$Z=X+Y$, where $X=\sum_{i=1}^N \pi_i \mathcal{N}(x;\mu^X_i,\Sigma^X_i)$ and $Y=\sum_{j=1}^N \phi_j \mathcal{N}(x;\mu^Y_j,\Sigma^Y_j)$ (*)

will be a new mixture distribution with $N^2$ component (where both $X, Y$ has $N$ components), the weights are the products of the old weights $\pi_i\phi_j$, and the distribution of the component $(i,j)$ is the distribution of the sum of independent random variables $X_i+Y_j$. So expanding that we are dealing now with: $$Z=\sum_{i=1}^N\sum_{j=1}^N\pi_i\phi_j(\mathcal{N}(x;\mu^X_i,\Sigma^X_i)+\mathcal{N}(x;\mu^Y_j,\Sigma^Y_j))= \sum_{i=1}^N\sum_{j=1}^N\pi_i\phi_j\mathcal{N}(x;\mu^X_i,\Sigma^X_i)+\pi_i\phi_j\mathcal{N}(x;\mu^Y_j,\Sigma^Y_j)$$ Now when we skip density parts, we can easily see that we have twice the sum of $\pi_i\phi_j$, where each of those sum to $1$, so in total our new weights sum to $2$. Of course we can generalize it to the sum of $M$ such models, which yields us a mixture model, where weights sum to $M$. Due to that, I have two questions:

1. Is this still a random variable as is? What I mean by that, is whether or not this new creation requires some normalization constant to be in $[0,1]$ interval. Based on that it seems to me, that normalization (by multiplication by $\frac{1}{M}$ where $M$ is the number of mixture components in sum) is strongly required, but I may be wrong. If yes, then this will again boil down to the new GMM model, which we can work on easily, with well-known techniques, if I am correct.
2. If not, is there a way to still use theory and properties/algorithms connected with regular GMM's with such creation? Structure of it seems to be very similar, if not the same, if we disregard constraint on weights summation. Due to that, it seems to be a pretty straightforward idea, to just normalize the weights and still use well developed and well-known techniques, but will it hold, theoretically?

(*) Note that here, we denote by $\mathcal{N}(x;\mu^Y_j,\Sigma^Y_j)$ we denote the density function of a multivariate gaussian variable, with mean $\mu_j^Y$ and covariance matrix $\Sigma_j^Y$

Answer 1

I think there may be a confusion in your reasonning.

When you write that $X = \sum_i \pi_i \mathcal{N}(\mu^X_i, \Sigma^X_i)$ you actually mean that the density of $X$ is a weighted average of normal densities : $f_X = \sum_i \pi_i \varphi_{\mu^X_i, \Sigma^X_i}$. But when you write that $Z = X + Y$ you mean that the random variable $Z$ is the sum of $X$ and $Y$ (thus the density of $Z$ is the convolution product of the densities of $X$ and $Y$)

When you write that $Z = \sum_i \sum_j \pi_i \phi_j (\mathcal{N}(\mu^X_i, \Sigma^X_i) + \mathcal{N}(\mu^Y_j, \Sigma^Y_j))$ the two uses of the summation are mixed. The $\sum_i ...$ refers to the summation of densities when the "$+$" between the $\mathcal{N}(...)$ refers to summation of random variables (and could be replaced by a convolution product).

So, from $Z = \sum_i \sum_j \pi_i \phi_j (\mathcal{N}(\mu^X_i, \Sigma^X_i) + \mathcal{N}(\mu^Y_j, \Sigma^Y_j))$ you get that, if the mixtures components of X and Y are independent, $$Z = \sum_i \sum_j \pi_i \phi_j (\mathcal{N}(\mu^X_i + \mu^Y_j, \Sigma^X_i + \Sigma^Y_j)$$ which is a mixture with $N^2$ components, and $\sum_i \pi_i \phi_i =1$ so all is good.

So a sum of mixture of gaussian is a mixture of gaussians if the sum of any component of $X$ and any component of $Y$ is still gaussian. In particular, this is the case when there are independent.