I currently read the book Finite Mixture and Markov Switching Models by Sylvia Frühwirth-Schnatter and I have a question regarding Section 6.2.1 regarding finite mixtures with normal components. Maybe someone can clarify my issues.

It is assumed that the observed univariate continuous data $y_1,...,y_N$ are i.i.d. realisations from a random variable $Y$, which is distributed as a $K$ component mixture distribution, where the components are univariate: \begin{align} p(y|\vartheta)=\eta_1 \cdot f_N(y;\mu_1,\sigma_1^2)+...+\eta_K \cdot f_N(y;\mu_K,\sigma_K^2) \end{align} where $f_N(y;\mu_k,\sigma_k^2) \sim \mathcal{N}(\mu_k,\sigma_k^2)$ is the density of a univariate normal-distributed component.

Parameter estimation for mixtures of normal-distributed components is now concerned with the estimation of the component parameters $(\mu_1,...,\mu_K)$ and $(\sigma_1,...,\sigma_K)$ as well as the weight parameters $(\eta_1,...\eta_K)$ for fixed $K$ and a data set $y=(y_1,...y_N)$ available which is assumed to be i.i.d distributed according to a mixture of normal distributions.

When the Allocations $S$ are known the posterior distribution of $\mu_k,\sigma_k^2$ given the complete data $S,y$ can be derived. The weights $(\eta_1,...,\eta_K)$ are known in this case as far as I understand if (correct me if I am wrong), because for every observation $y_i \in (y_1,...,y_N)$ it is known to which group $y_i$ belongs, that is, the quantities $S_i=k, k\in \{1,...K\}$ are available for all $i\in \{1,...,N\}$.

I skip most derivations here for now, and the main result is, that the joint posterior can be written as \begin{align} p(\mu,\sigma^2|S,y)=p(\mu_1,...,\mu_K,\sigma_1^2,...,\sigma_K^2|S,y)=\prod_{k=1}^{K}p(\mu_k,\sigma_k^2|S,y)=\prod_{k=1}^{K}\underbrace{p(\mu_k|\sigma_k^2,S,y)}_{=:(A)}\cdot \underbrace{p(\sigma_k^2|S,y)}_{:=(B)} \end{align}

where the factor (A) of the posterior is normal-distributed $\mathcal{N}(b_k(S),B_k(S))$ with parameters \begin{align} &B_k(S)=\frac{1}{N_0+N_k(S)}\sigma_k^2\\ &b_k(S)=\frac{N_0}{N_k(S)+N_0}b_0+\frac{N_k(S)}{N_k(S)+N_0}\bar{y}_k(S) \end{align}

and the factor $(B)$, which is the marginal posterior of $\sigma_k^2$, is distributed as $\mathcal{G}^{-1}(c_k(S),C_k(S))$, following an inverse-Gamma distribution with parameters \begin{align} &c_k(S)=c_0+\frac{1}{2}N_k(S)\\ &C_k(S)=C_0+\frac{1}{2}\left ( N_k(S) s_{y,k}^2(S)+\frac{N_k(S) N_0}{N_k(S)+N_0}(\bar{y}_k(S)-b_0)^2 \right ) \end{align}

Now, my question is:

In the joint posterior \begin{align} p(\mu,\sigma^2|S,y)=\prod_{k=1}^{K}\underbrace{p(\mu_k|\sigma_k^2,S,y)}_{=:(A)}\cdot \underbrace{p(\sigma_k^2|S,y)}_{:=(B)} \end{align}

we have for example for $K=2$ a product of two sub products, each of which consists of a normal density multiplied by an inverse gamma density with some parameters. The section in the book then closes by stating that this is a closed-form posterior. I do not understand, how I could directly simulate this distribution now. Does this product of densities result in another density, and if this is the case, can anybody tell me which one?

Thanks in advance,



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.