2
$\begingroup$

I came across the following which I have troubles understanding and would appreciate your help.

Suppose we want to fit a Gaussian mixture model (GMM) to a $t$-dstribution with some degrees of freedom. Let the GMM have $K$ components. Using the $EM$-algorithm we found $K$ means $\mu_k$ and variances $\sigma^2_k$ of the Gaussians as well as $K$ mixing coefficients $\pi_k$.

In order to sample from the GMM we use an ancestral sampling scheme with one exception (steps 4 and 5). That is, for one sample, we have to

  1. generate $U$ following a standard uniform distribution.
  2. compute $k=\sum_{j=1}^K I_{\{g_j>U\}} + 1$, with $g_j=\sum_{i=1}^j\pi_i$, $j=1,...,K$.
  3. Given $k$ choose $\mu_k$ and $\sigma^2_k$.
  4. Sample $v_t$, $t=1,...,L$, each following to $N(\mu_k, \sigma^2_k)$.
  5. Compute $V=\sum_{t=1}^Lv_t$

Note that in step 4 we sample from the same Gaussian $L$ samples which are summed in 5.

Now the claim is that $V/\sqrt{L}$ is GMM-distributed, where the scaling comes from the sum of $L$ Gaussians. In this particular case approximately $t$-distributed. Is this true? I find it odd but cannot grasp why. Thanks for help.

$\endgroup$
2
$\begingroup$

Given that you are taking the sum of $L$ Gaussian $\mathcal{N}(\mu_k,\sigma^2_k)$ variates, the sum $V$ is also Gaussian $\mathcal{N}(L\mu_k,L^2\sigma^2_k)$. Hence your outcome $V/\sqrt{L}$ is still one from a Gaussian mixture $$\sum_{k=1}^K \pi_k \mathcal{N}(\sqrt{L}\mu_k,L\sigma^2_k)$$ I must add that I see little motivation for considering this simulation and average.

$\endgroup$
0
$\begingroup$

Thanks for the answer, I appreciate it very much. I agree with your comment regarding the motivation.

Regarding your answer: Isn't the variance of the variance of $V$ equal to $\sum_{k=1}^K \omega_k \left(\sigma_k^2+L(\mu_k-\mu_g)^2\right)$?, with, $\mu_g=\sum_k\omega_k\mu_k$?

Here is how I derived it, with $v_{t,k}$ denoting the random variables form the $K$ components:

$$\sum_k\omega_k E\left(\frac{\sum_t v_{t,k}}{\sqrt{L}}-\sqrt{L}\mu_g\right)^2\\ = \frac{1}{L}\sum_k\omega_kE\left(\sum_t v_{t,k}-L\mu_k+L\mu_k-L\mu_g \right)^2\\ =\frac{1}{L}\sum_k\omega_k\left(E\left(\sum_t (v_{t,k}-\mu_k)\right)^2+2E\left(\sum_tv_{t,k}-L\mu_k\right)\left(L\mu_k-L\mu\right)+L^2\left(\mu_k-\mu_g\right)^2\right)\\ =\frac{1}{L}\sum_k\omega_k\left(L\sigma_k^2+L^2(\mu_k-\mu_g)^2\right)\\ =\sum_k\omega_k\left(\sigma_k^2+L(\mu_k-\mu_g)^2\right) $$

The middel term in the longest line vanishes. Where is the mistake? Thanks.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.