What will be the distribution of the sum of two independent random variables, say $X$ and $Y$, when $X$ has a Gaussian mixture distribution (when we take Gaussian distribution with different location parameters), and $Y$ is a Gaussian scale mixture (when we take Gaussian distributions with different scale parameters)?

Let $Z=X+Y$ then, the mixture distribution of $X$ has density $\sum_{i=1}^n \pi_i \mathcal{N}(x;\mu_i,\sigma^2_X)$ and the mixture distribution of $Y$ has density $\sum_{j=1}^m \phi_j \mathcal{N}(x;\mu_Y,\sigma^2_j)$, the symbols $\mathcal{N}(x, \dots)$ being normal densities. (One could as well let both parameters vary in both cases, $X$ and $Y$).

  • 1
    $\begingroup$ Could you explain more clearly what you mean by, "sum of Gaussian mixture"? $\endgroup$ – conjectures Sep 30 '15 at 9:48
  • $\begingroup$ Let $Y=X+Z$ where X is just a sum of Gaussians (where each Gaussian is multiplied by the respective mixing coefficient) i.e it is sum of Gaussian with different location parameter. While Z is sum of Gaussian with different scale parameter. I want to know what will be sum of X and Z ie What will be distribution of Y. $\endgroup$ – undefined Sep 30 '15 at 9:56
  • 1
    $\begingroup$ This is unclear. Write down the distributions of $X$ and $Y$ in the question, not in the comments. $\endgroup$ – Xi'an Sep 30 '15 at 13:49
  • 5
    $\begingroup$ I will edit the question as I think is your suggestion. If that is wrong, protest! $\endgroup$ – kjetil b halvorsen Oct 2 '15 at 14:06
  • $\begingroup$ Thanks for the edit I meant this only. I think I need to learn the skills to ask question $\endgroup$ – undefined Oct 3 '15 at 12:37

Let $X$ has the mixture distribution with density $f(x)=\sum \pi_i f_i(x)$ and $Y$ the mixture distribition with density $g(x) = \sum \phi_i g_i(x)$, and suppose $X$ and $Y$ are independent. A useful tool for analyzing distribution of sums of independent random variables is the moment generating function (look it upon wikipedia if you didn't see it yet). That is given by $\DeclareMathOperator{\E}{E} M_X(t) = \E e^{tX}$. Then the moment generating function of $X+Y$ is the product $M_X(t) M_Y(t)$. Let the moment generating functions for $X$ mixture component $i$ be $M_i$, for $Y$ mixture component $j$ be $G_j$. Then, by linearity of the expectation operator, we have $$ M_X(t) = \E e^{tX} = \int e^{tx} f(x)\; dx \\ = \int e^{tx} \sum_i \pi_i f_i(x)\; dx \\ = \sum_i \pi_i M_i(t) $$ and likewise for $Y$ $G_Y(t) = \sum_j G_j(t)$ and then the moment generating function for $X+Y$ is $$ M_X(t)G_Y(t)=\sum_i \pi_i M_i(t) \cdot \sum_j \phi_j G_j(t) \\ = \sum_i \sum_j \pi_i \phi_j M_i(t) G_j(t) $$ so the distribution of the sum is a new mixture distribution with $nm$ component (where $X$ has $n$ components, $Y$ has $m$ components), the weights are the products of the old weights $\pi_i \phi_j$, and the distribution of the component $(ij)$ is the distribution of the sum of independent random variables $X_i+Y_j$.

In your case all of those will be normal distributions, but I leave for you to work out the details. Observe that only the very last step of the argument (that left for you) pends on thse being a normal mixture. So what we have shown is a quite gneral result about the istribution of sums of random variables, each one with a mixture distribution.


If I understand your notation you are asking about the distribution of $Y$ when $Y = X + Z$ where $X \sim \mathcal N(a,b)$, $Z \sim \mathcal N(c,d)$.

In this case $Y$ is Gaussian if $Z$ and $X$ are independent or jointly normal, with mean $a+b$ and a covariance matrix dependent on the relationship between $X$ and $Z$.

Using the word mixture to describe the above arrangement will lead to confusion.

  • $\begingroup$ The term mixture is important as X and Z are not Gaussian. X and Z belong to Gaussian mixture and Gaussian scale mixture distribution. We can say X is weighted sum of Gaussian density with different location parameter as $ X=\sum_{i=1}^N w_i N(\mu_i,\Sigma) $ and Z is weighted sum of Gaussian with different scale parameter. $Z=dU$ where $U \sim \mathdd{N}(0,Q)$ and $d$ is scalar random variable with density as mixing distribution. $\endgroup$ – undefined Sep 30 '15 at 12:47
  • 2
    $\begingroup$ Try being more clear with your descriptions. You said above "X is just a sum of Gaussians." That is not a mixture. You've picked a bad choice of notation if you wish $X$, $Y$ and $Z$ to represent distribution functions rather than random variables and want people to implicitly guess that. $\endgroup$ – conjectures Sep 30 '15 at 13:19
  • 1
    $\begingroup$ Writing $X=\sum_{i=1}^N w_i N(\mu_i,\Sigma)$ perpetuates the confusion. Do you mean $X\sim\sum_{i=1}^N w_i N(\mu_i,\Sigma)$ as in a mixture model? Or $X=\sum_{i=1}^N w_i X_i$ with $X_i\sim N(\mu,\Sigma)$ as in a Gaussian weighted average? $\endgroup$ – Xi'an Sep 30 '15 at 13:50
  • $\begingroup$ No X is mixture model ie it is weighted sum of Gaussian densities. not weighted sum of Gaussian random variable hope I am clear now. $\endgroup$ – undefined Oct 1 '15 at 5:59
  • 1
    $\begingroup$ This is impenetrable and I thus vote to close this question as "unclear". $\endgroup$ – Xi'an Oct 1 '15 at 11:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.