What are Gaussian scale mixture? Is it different from Gaussian mixture.

What is overall location and scale parameter of given Gaussian scale mixture and how to generate a samples of given $\mu$ and $\sigma^2$.


2 Answers 2


The normal (or gaussian) pdf (probability density function) is $$ \DeclareMathOperator{\E}{E} \DeclareMathOperator{\Var}{Var} f(x;\mu,\sigma^2) = \frac1{\sqrt{2\pi\sigma^2}}\cdot \exp\left(-\frac12 (\frac{x-\mu}{\sigma})^2 \right) $$ Here $\mu$ is the location parameter (mean) and $\sigma^2$ is the scale parameter (variance). A Gaussian mixture is usually when we take Gaussian distribution with different location parameters, but a scale mixture refers to the case with varying scale parameters, so is given by (a discrete mixture) $$ \sum_{i=1}^n \pi_i \cdot f(x; \mu, \sigma_i^2) $$ where the weights $\pi_i$ are non-negative and sums to 1. We can also have a continuous mixture $$ \int_0^\infty g(\sigma^2) f(x;\mu,\sigma^2) \; d\sigma^2 $$ where $g(\sigma^2)$ is a density function.

One could also have mixtures where both parameters are varying. There are some Cross Validated posts about estimating mixture distributions from data, see https://stats.stackexchange.com/search?q=estimating+mixture+distributions

The question about expectation and variance of the mixture distribution: In the scale mixture, all the components have the same expectation, so that will be the expectation of the mixture also. That can be seen formally by using the double expectation theorem as below. We will look at the general case, where both parameters can vary, so we can write the mixture distribution as $X | I \sim \text{N}(\mu_i, \sigma^2_i)$ where the conditioning variable $I$ have the distribution $\{ \pi_i\}$ in discrete case, $g(\cdot)$ in continuous case. In the general case, $g(\cdot)$ must be a density in both $\mu$ and $\sigma^2$, we indicate which joint/marginal we are using by the arguments. Then the expectation of the mixture becomes (since the varying variances do not contribute anything to the expectation) $$ \mu = \E X = \E [\E X|I] =\begin{cases} \sum \pi_i \mu_i &~ \text{discrete case} ~ \\ \int_{-\infty}^\infty \mu g(\mu) \; d \mu &~ \text{continuous case} ~ \end{cases} $$ For the variance we need the double variance theorem, which is $\Var X = \E \Var X|I + \Var \E X|I$, which gives: $$ \Var X = \E \Var X|I + \Var \E X|I = \sum \pi_i \sigma^2_i + \sum \pi_i (\mu_i-\mu)^2 $$ and I leave the continuous case as an exercise.

So for random generation: The mixture distribution can be represented stochastically by (as above) $$ X | I=i \sim \text{N}(\mu_i, \sigma^2_i) $$ and then we must specify the distribution of $I$. For the discrete case, say $P(I=i)=\pi_i, i=1,2\dots,n$.

Then the simulation follows this, but in the opposite order: We start with $I$, simulate that, and conditional on the result, we simulate $X|I=i$. For a concrete example, lets say $I$ has a Poisson distribution with parameter (mean) $\lambda=1$, and $X|I=i \sim \text{N}(\mu=i, \sigma^2=i^2)$. We can implement this in R as follows:

> I <- rpois(10000,1.0)
> X  <- rnorm(10000,I,I)
> hist(X,prob=TRUE)

the histogram shown below:

enter image description here

  • $\begingroup$ What will be the density of $g(\sigma^2)$? And how can we find weights. $\endgroup$
    – undefined
    Commented Sep 28, 2015 at 10:55
  • 2
    $\begingroup$ You are the one defining your mixture, so you choose the weights, or maybe, estimate them from data. I will add something about that to the answer $\endgroup$ Commented Sep 28, 2015 at 10:57
  • $\begingroup$ Then what will be the overall scale and location parameter of given samples? I mean if i want to generate multivariate samples of Gaussian scale mixture with $\mu=0$ and $\sigma^2$, how will I generate that? $\endgroup$
    – undefined
    Commented Sep 28, 2015 at 11:01
  • 1
    $\begingroup$ Given you've given a good answer to an existing question, it might be better if @undefined posted a new question, but if you're both happy with this question being updated, that's okay. $\endgroup$
    – Glen_b
    Commented Sep 28, 2015 at 11:06
  • 1
    $\begingroup$ I will add more extensive explication! $\endgroup$ Commented Sep 29, 2015 at 8:58

A Gaussian scale mixture (not to be confused with a mixture of gaussians) is a random variable $Y$ that can be expressed as the following mixture distribution $$ Y|(Z=z) \sim N(\mu,z^{-1}\Sigma) $$ where Z is a random variable with positive domain. For more details see these papers



One example is when $Z\sim gamma(d/2,d/2)$, then we have the multivariate $t$ distribution with $d$ degrees of freedom. To simulate from these distributions you first simulate one observation $z$ from $Z$ and then simulate your variable from $N(\mu,z^{-1}\Sigma)$.

A gaussian mixture is what kjetil b halvorsen answered


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.