2
$\begingroup$

I know that the likelihood in a p-dimensional Gaussian mixture model is given by

$$ p(s|\theta) = \sum_{b_1 = 0}^1\cdots\sum_{b_p = 0}^1\left[ \prod_{i=1}^pw^{1-b_i}(1-w)^{1-b_i}\right]\phi_p(s|\mu(b,\theta),\Sigma) $$

where $\phi_p(x|a,B)$ denota a p-dimensional Gaussian density function, $w \in [0,1]$ is a mixture weight, $\mu(b,\theta)' = ((1-2b_1)\theta_1, \cdots, (1-2b_p)\theta_p)$, $b_i \in \{0,1\}$ and $\Sigma = [\Sigma_{i,j}]$ is such that $\Sigma_{i,i} = 1$ and $\Sigma_{i,j} = \rho$ for $i \neq j$.

For $p = 2$, let $$s \sim N(\mu, \Sigma)$$ $$\mu_i = (1-2b_i)\theta_i$$ $$b_i \sim Bern(1-w)$$ $$\theta_i \sim U(-20, 40)$$

You can demonstrate that the full conditional distribution for $\theta_1$ and $b_1$, respectively, are given by

$$p(\theta_1|\theta_2,b,s) \sim N\left(\mu_{\theta_1}, \sqrt{1-\rho^2}\right)$$

where $\mu_{\theta_1} = [\rho((1-2b_2)\theta_2 - s_2 + s_1)/(1-2b_1)]$.

In the model above, replacing $\theta_1$ with $u_1 + u_2$ and $\theta_2$ with $u_1 - u_2$, where $u_1, u_2 \sim U(-20, 40)$ can you still guarantee normality?

I don't know if I'm misinterpreting, but when replacing a uniform with a (triangular) uniform sum, would the convergence to a normal distribution not have to be faster? I did some simulation studies and did not get a satisfactory result with this hypothesis. Thanks in advance!


I got the following answer:

$$\theta_1, \theta_2 \sim U(-20,40) \quad \theta_1\perp \theta_2$$

Under the assumption of independence between,

$$f_{\theta_1, \theta_2}(\theta_1, \theta_2) = \frac{1}{3600}\mathbb{I}_{\{-20\leq \theta_1 \leq 40, -20 \leq \theta_2 \leq 40\}}(\theta_1, \theta_2)$$

Making variable changes

$\begin{cases} \alpha_1 = g_1(\theta_1, \theta_2) = \theta_1 + \theta_2\\ \alpha_2 = g_2(\theta_1, \theta_2) = \theta_1 - \theta_2 \end{cases} \Rightarrow \begin{cases} \theta_1 = \frac{\alpha_1 + \alpha_2}{2}\\ \theta_2 = \frac{\alpha_1 - \alpha_2}{2}\\ \end{cases} \qquad |J| = \frac{1}{2}$

$$\begin{align*} f_(\alpha_1, \alpha_2) = & \:f_{\theta_1, \theta_2}(g_1^{-1}, g_2^{-1})|J|\\ = & \:\frac{1}{3600}\mathbb{I}_{\left\{-20\leq \frac{\alpha_1 + \alpha_2}{2} \leq 40, -20\leq \frac{\alpha_1 - \alpha_2}{2} \leq 40 \right\}}(\alpha_1, \alpha_2)\frac{1}{2} \end{align*}$$

The variation set follows as below.

$$A = \left\{(\alpha_1, \alpha_2);\:-20\leq \frac{\alpha_1 + \alpha_2}{2} \leq 40, -20\leq \frac{\alpha_1 - \alpha_2}{2} \leq 40 \right\}$$

The marginal of $\alpha_1$ is given by

$\begin{align*} f_{\alpha_1}(\alpha_1) = & \: \int_{-\infty}^\infty f(\alpha_1, \alpha_2)\:d\alpha_2\\ = & \: \begin{cases} \frac{\alpha_1 + 40}{3600},\: -40\leq \alpha_1 \leq 20\\ \frac{80 - \alpha_1}{3600},\: 20\leq \alpha_1 \leq 80\\ \end{cases} \end{align*}$

Now

$$\begin{align*} p(\alpha_1 | \alpha_2, b, s) \propto & \: p(s | \alpha, b, s) p(\alpha_1)\\ = & \: \begin{cases} \phi_2(s | \mu(b, \alpha),\Sigma) \frac{\alpha_1 + 40}{3600},\: -40 \leq \alpha_1 \leq 20\\ \phi_2(s | \mu(b, \alpha),\Sigma) \frac{80 - \alpha_1}{3600},\: 20 \leq \alpha_1 \leq 80 \end{cases} \end{align*}$$

That when solving $p(\alpha_1 | \alpha_2, b, s)$ in $\alpha_1$, note that the result will be as follows $$f(\alpha_1)e^{-g(\alpha_1)}$$ That runs away from the core of a normal.

Is there any misconception?

$\endgroup$
  • 1
    $\begingroup$ You didn't specify the prior for $w$ or $\Sigma$ $\endgroup$ – Taylor Sep 14 at 14:05
  • 1
    $\begingroup$ I presume the inner likelihood term should be$$\prod_i\omega^{b_i}(1-\omega)^{1-b_i}$$ $\endgroup$ – Xi'an Sep 14 at 14:39
  • 1
    $\begingroup$ The full conditional distribution of $\theta_1$ is restricted to $(-20,40)$ hence a truncated Normal at best. $\endgroup$ – Xi'an Sep 14 at 14:45
  • $\begingroup$ @Taylor, I believe the above model is already well specified. For example, $s' = (2,2), w = 0,3$ and $\rho = 0,3$. are values for these parameters. $\endgroup$ – Jackson Maike Sep 14 at 15:23
  • $\begingroup$ @Xi'an, the post from which I took the model points out normality when I wear a uniform for $\theta$. My idea is that if I use a $f(\theta)$ in which its distribution is symmetrical, such as triangular, normality would still have to be maintained. Am I right? Anyway I will try to demonstrate analytically. $\endgroup$ – Jackson Maike Sep 14 at 15:27
0
$\begingroup$

This is a very peculiar kind of mixture distribution in that, for each component $s_i$ $(i=1,\ldots,p)$ of a random realisation $\mathbf s=(s_1,\ldots,s_p)$ from this distribution, the mean is either $\theta_i$ or $-\theta_i$ with probabilities $\omega$ and $1-\omega$, independently. Formally, this makes a mixture with $2^p$ components. If $\boldsymbol \theta$ is the only (unknown) parameter of the model the posterior distribution of $\boldsymbol \theta$ conditional on one single observation $\mathbf s$ and the associated allocation latent variable $\mathbf b=(b_1,\ldots,b_p)\in\{-1,1\}^p$ is the product of the prior on $\boldsymbol \theta$ by a Normal likelihood $\varphi(\mathbf s|\mathbf b\cdot\boldsymbol \theta,\Sigma)$ (where $\mathbf b\cdot\boldsymbol \theta$ denotes a component-wise product).

Were the prior to be flat, the posterior would then be a Normal distribution $\varphi(\boldsymbol \theta|\mathbf b\cdot\mathbf s,\mathbf b^\prime\cdot\Sigma\cdot\mathbf b)$ since \begin{align*}(\mathbf s-\mathbf b\cdot\boldsymbol \theta)^\prime\Sigma^{-1}(\mathbf s-\mathbf b\cdot\boldsymbol \theta)&=(\mathbf b\cdot\mathbf b\cdot\mathbf s-\mathbf b\cdot\boldsymbol \theta)^\prime\Sigma^{-1}(\mathbf b\cdot\mathbf b\cdot\mathbf s-\mathbf b\cdot\boldsymbol \theta)\\&=(\mathbf b\cdot\mathbf s-\boldsymbol \theta)^\prime\mathbf b^\prime\cdot\Sigma^{-1}\cdot\mathbf b(\mathbf b\cdot\mathbf s-\boldsymbol \theta)\end{align*} but since the prior is a Uniform on $(-20,40)^p$, the posterior is a truncated Normal and hence the posterior is then a truncated Normal distribution.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.