4
$\begingroup$

What are Gaussian scale mixture? Is it different from Gaussian mixture.

What is overall location and scale parameter of given Gaussian scale mixture and how to generate a samples of given $\mu$ and $\sigma^2$.

$\endgroup$
7
$\begingroup$

The normal (or gaussian) pdf (probability density function) is $$ \DeclareMathOperator{\E}{E} \DeclareMathOperator{\Var}{Var} f(x;\mu,\sigma^2) = \frac1{\sqrt{2\pi\sigma^2}}\cdot \exp\left(-\frac12 (\frac{x-\mu}{\sigma})^2 \right) $$ Here $\mu$ is the location parameter (mean) and $\sigma^2$ is the scale parameter (variance). A Gaussian mixture is usually when we take Gaussian distribution with different location parameters, but a scale mixture refers to the case with varying scale parameters, so is given by (a discrete mixture) $$ \sum_{i=1}^n \pi_i \cdot f(x; \mu, \sigma_i^2) $$ where the weights $\pi_i$ are non-negative and sums to 1. We can also have a continuous mixture $$ \int_0^\infty g(\sigma^2) f(x;\mu,\sigma^2) \; d\sigma^2 $$ where $g(\sigma^2)$ is a density function.

One could also have mixtures where both parameters are varying. There are some Cross Validated posts about estimating mixture distributions from data, see https://stats.stackexchange.com/search?q=estimating+mixture+distributions

The question about expectation and variance of the mixture distribution: In the scale mixture, all the components have the same expectation, so that will be the expectation of the mixture also. That can be seen formally by using the double expectation theorem as below. We will look at the general case, where both parameters can vary, so we can write the mixture distribution as $X | I \sim \text{N}(\mu_i, \sigma^2_i)$ where the conditioning variable $I$ have the distribution $\{ \pi_i\}$ in discrete case, $g(\cdot)$ in continuous case. In the general case, $g(\cdot)$ must be a density in both $\mu$ and $\sigma^2$, we indicate which joint/marginal we are using by the arguments. Then the expectation of the mixture becomes (since the varying variances do not contribute anything to the expectation) $$ \mu = \E X = \E [\E X|I] =\begin{cases} \sum \pi_i \mu_i &~ \text{discrete case} ~ \\ \int_{-\infty}^\infty \mu g(\mu) \; d \mu &~ \text{continuous case} ~ \end{cases} $$ For the variance we need the double variance theorem, which is $\Var X = \E \Var X|I + \Var \E X|I$, which gives: $$ \Var X = \E \Var X|I + \Var \E X|I = \sum \pi_i \sigma^2_i + \sum \pi_i (\mu_i-\mu)^2 $$ and I leave the continuous case as an exercise.

So for random generation: The mixture distribution can be represented stochastically by (as above) $$ X | I=i \sim \text{N}(\mu_i, \sigma^2_i) $$ and then we must specify the distribution of $I$. For the discrete case, say $P(I=i)=\pi_i, i=1,2\dots,n$.

Then the simulation follows this, but in the opposite order: We start with $I$, simulate that, and conditional on the result, we simulate $X|I=i$. For a concrete example, lets say $I$ has a Poisson distribution with parameter (mean) $\lambda=1$, and $X|I=i \sim \text{N}(\mu=i, \sigma^2=i^2)$. We can implement this in R as follows:

> I <- rpois(10000,1.0)
> X  <- rnorm(10000,I,I)
> hist(X,prob=TRUE)

the histogram shown below:

enter image description here

$\endgroup$
  • $\begingroup$ What will be the density of $g(\sigma^2)$? And how can we find weights. $\endgroup$ – undefined Sep 28 '15 at 10:55
  • 2
    $\begingroup$ You are the one defining your mixture, so you choose the weights, or maybe, estimate them from data. I will add something about that to the answer $\endgroup$ – kjetil b halvorsen Sep 28 '15 at 10:57
  • $\begingroup$ Then what will be the overall scale and location parameter of given samples? I mean if i want to generate multivariate samples of Gaussian scale mixture with $\mu=0$ and $\sigma^2$, how will I generate that? $\endgroup$ – undefined Sep 28 '15 at 11:01
  • $\begingroup$ This is new questions which should really be an edit to your original post, please do that! I will try to answer in short time. $\endgroup$ – kjetil b halvorsen Sep 28 '15 at 11:03
  • $\begingroup$ Given you've given a good answer to an existing question, it might be better if @undefined posted a new question, but if you're both happy with this question being updated, that's okay. $\endgroup$ – Glen_b Sep 28 '15 at 11:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.