3
$\begingroup$

I know that the likelihood in a p-dimensional Gaussian mixture model is given by

$$ p(s|\theta) = \sum_{b_1 = 0}^1\cdots\sum_{b_p = 0}^1\left[ \prod_{i=1}^pw^{1-b_i}(1-w)^{1-b_i}\right]\phi_p(s|\mu(b,\theta),\Sigma) $$

where $\phi_p(x|a,B)$ denota a p-dimensional Gaussian density function, $w \in [0,1]$ is a mixture weight, $\mu(b,\theta)' = ((1-2b_1)\theta_1, \cdots, (1-2b_p)\theta_p)$, $b_i \in \{0,1\}$ and $\Sigma = [\Sigma_{i,j}]$ is such that $\Sigma_{i,i} = 1$ and $\Sigma_{i,j} = \rho$ for $i \neq j$.

For $p = 2$, let $$s \sim N(\mu, \Sigma)$$ $$\mu_i = (1-2b_i)\theta_i$$ $$b_i \sim Bern(1-w)$$ $$\theta_i \sim U(-20, 40)$$

You can demonstrate that the full conditional distribution for $\theta_1$ and $b_1$, respectively, are given by

$$p(\theta_1|\theta_2,b,s) \sim N\left(\mu_{\theta_1}, \sqrt{1-\rho^2}\right)$$

where $\mu_{\theta_1} = [\rho((1-2b_2)\theta_2 - s_2 + s_1)/(1-2b_1)]$.

In the model above, replacing $\theta_1$ with $u_1 + u_2$ and $\theta_2$ with $u_1 - u_2$, where $u_1, u_2 \sim U(-20, 40)$ can you still guarantee normality?

I don't know if I'm misinterpreting, but when replacing a uniform with a (triangular) uniform sum, would the convergence to a normal distribution not have to be faster? I did some simulation studies and did not get a satisfactory result with this hypothesis. Thanks in advance!


I got the following answer:

$$\theta_1, \theta_2 \sim U(-20,40) \quad \theta_1\perp \theta_2$$

Under the assumption of independence between,

$$f_{\theta_1, \theta_2}(\theta_1, \theta_2) = \frac{1}{3600}\mathbb{I}_{\{-20\leq \theta_1 \leq 40, -20 \leq \theta_2 \leq 40\}}(\theta_1, \theta_2)$$

Making variable changes

$\begin{cases} \alpha_1 = g_1(\theta_1, \theta_2) = \theta_1 + \theta_2\\ \alpha_2 = g_2(\theta_1, \theta_2) = \theta_1 - \theta_2 \end{cases} \Rightarrow \begin{cases} \theta_1 = \frac{\alpha_1 + \alpha_2}{2}\\ \theta_2 = \frac{\alpha_1 - \alpha_2}{2}\\ \end{cases} \qquad |J| = \frac{1}{2}$

$$\begin{align*} f_(\alpha_1, \alpha_2) = & \:f_{\theta_1, \theta_2}(g_1^{-1}, g_2^{-1})|J|\\ = & \:\frac{1}{3600}\mathbb{I}_{\left\{-20\leq \frac{\alpha_1 + \alpha_2}{2} \leq 40, -20\leq \frac{\alpha_1 - \alpha_2}{2} \leq 40 \right\}}(\alpha_1, \alpha_2)\frac{1}{2} \end{align*}$$

The variation set follows as below.

$$A = \left\{(\alpha_1, \alpha_2);\:-20\leq \frac{\alpha_1 + \alpha_2}{2} \leq 40, -20\leq \frac{\alpha_1 - \alpha_2}{2} \leq 40 \right\}$$

The marginal of $\alpha_1$ is given by

$\begin{align*} f_{\alpha_1}(\alpha_1) = & \: \int_{-\infty}^\infty f(\alpha_1, \alpha_2)\:d\alpha_2\\ = & \: \begin{cases} \frac{\alpha_1 + 40}{3600},\: -40\leq \alpha_1 \leq 20\\ \frac{80 - \alpha_1}{3600},\: 20\leq \alpha_1 \leq 80\\ \end{cases} \end{align*}$

Now

$$\begin{align*} p(\alpha_1 | \alpha_2, b, s) \propto & \: p(s | \alpha, b, s) p(\alpha_1)\\ = & \: \begin{cases} \phi_2(s | \mu(b, \alpha),\Sigma) \frac{\alpha_1 + 40}{3600},\: -40 \leq \alpha_1 \leq 20\\ \phi_2(s | \mu(b, \alpha),\Sigma) \frac{80 - \alpha_1}{3600},\: 20 \leq \alpha_1 \leq 80 \end{cases} \end{align*}$$

That when solving $p(\alpha_1 | \alpha_2, b, s)$ in $\alpha_1$, note that the result will be as follows $$f(\alpha_1)e^{-g(\alpha_1)}$$ That runs away from the core of a normal.

Is there any misconception?

$\endgroup$
6
  • 1
    $\begingroup$ You didn't specify the prior for $w$ or $\Sigma$ $\endgroup$
    – Taylor
    Sep 14, 2019 at 14:05
  • 1
    $\begingroup$ I presume the inner likelihood term should be$$\prod_i\omega^{b_i}(1-\omega)^{1-b_i}$$ $\endgroup$
    – Xi'an
    Sep 14, 2019 at 14:39
  • $\begingroup$ @Taylor, I believe the above model is already well specified. For example, $s' = (2,2), w = 0,3$ and $\rho = 0,3$. are values for these parameters. $\endgroup$
    – jassis
    Sep 14, 2019 at 15:23
  • $\begingroup$ @Xi'an, the post from which I took the model points out normality when I wear a uniform for $\theta$. My idea is that if I use a $f(\theta)$ in which its distribution is symmetrical, such as triangular, normality would still have to be maintained. Am I right? Anyway I will try to demonstrate analytically. $\endgroup$
    – jassis
    Sep 14, 2019 at 15:27
  • $\begingroup$ Given the peculiarity of the model (which is not a standard mixture model since the number of components grows exponentially in $p$), could you indicate a relevant reference or a motivation for this model? $\endgroup$
    – Xi'an
    Sep 15, 2019 at 14:21

1 Answer 1

2
$\begingroup$

This is a very peculiar kind of mixture distribution in that, for each component $s_i$ $(i=1,\ldots,p)$ of a random realisation $\mathbf s=(s_1,\ldots,s_p)$ from this distribution, the mean is either $\theta_i$ or $-\theta_i$ with probabilities $\omega$ and $1-\omega$, independently. Formally, this makes a mixture with $2^p$ components. If $\boldsymbol \theta$ is the only (unknown) parameter of the model the posterior distribution of $\boldsymbol \theta$ conditional on one single observation $\mathbf s$ and the associated allocation latent variable $\mathbf b=(b_1,\ldots,b_p)\in\{-1,1\}^p$ is the product of the prior on $\boldsymbol \theta$ by a Normal likelihood $\varphi(\mathbf s|\mathbf b\cdot\boldsymbol \theta,\Sigma)$ (where $\mathbf b\cdot\boldsymbol \theta$ denotes a component-wise product).

Were the prior to be flat, the posterior would then be a Normal distribution $\varphi(\boldsymbol \theta|\mathbf b\cdot\mathbf s,\mathbf b^\prime\cdot\Sigma\cdot\mathbf b)$ since \begin{align*}(\mathbf s-\mathbf b\cdot\boldsymbol \theta)^\prime\Sigma^{-1}(\mathbf s-\mathbf b\cdot\boldsymbol \theta)&=(\mathbf b\cdot\mathbf b\cdot\mathbf s-\mathbf b\cdot\boldsymbol \theta)^\prime\Sigma^{-1}(\mathbf b\cdot\mathbf b\cdot\mathbf s-\mathbf b\cdot\boldsymbol \theta)\\&=(\mathbf b\cdot\mathbf s-\boldsymbol \theta)^\prime\mathbf b^\prime\cdot\Sigma^{-1}\cdot\mathbf b(\mathbf b\cdot\mathbf s-\boldsymbol \theta)\end{align*} but since the prior is a Uniform on $(-20,40)^p$, the posterior is a truncated Normal and hence the posterior is then a truncated Normal distribution.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.