1
$\begingroup$

Let $\theta_i$ be an indicator which is $0$ if score, $X_i$, is the same for both opponents, $1$ if different:

  • $X_i|\theta_i \stackrel{\text{ind}}{\sim} (1-\theta_i) U(0, 1) + \theta_i Beta(1, \beta)$
  • $\theta_i \stackrel{\text{iid}}{\sim}\text{Bernoulli}(\pi)$
  • $\pi \sim \text{Uniform}(0,1)$
  • $\beta \sim \text{Gamma}(\text{shape} = 0.001, \text{rate} = 0.001)$

Attempt: We need to find the following conditional posteriors which I am having trouble with. I have came across combining normal distributions which I understand, what is the approach for these more complicated distributions (any references would also be appreciated)?

$$\theta_i|X_i \sim $$ $$\pi|X_i \sim $$ $$\beta|X_i \sim $$

$\endgroup$
  • $\begingroup$ The Gibbs sampler for mixtures is one of the first examples of applying the Gibbs sampler and a straightforward one. Can you tell us where you get stuck in deriving the joint distribution of $\pi,\beta,\theta_1,\ldots,\theta_n$? $\endgroup$ – Xi'an Dec 13 '15 at 11:20
1
$\begingroup$

Here is how we explain the resolution in Bayesian Essentials with R rephrased for your problem (but you can find the same approach in most textbooks):

With each $x_i$ is associated a missing variable $z_i$ [your $\theta_i$] that indicates its component. Formally, this means that we have a hierarchical structure associated with the model: $$ z_i|\pi\sim\mathscr{B}(\pi) $$ and $$ x_i|z_i=1,\beta \sim \beta x^{\beta-1}\qquad x_i|z_i=0,\beta \sim \mathcal{U}(01,1) $$ The completed likelihood corresponding to the missing structure is such that $$ \ell(\beta,\pi|\mathbf{x},\mathbf{z})=\prod_{i=1}^n \pi^{z_i}(1-\pi)^{1-z_i}\,(1-x_i)^{(\beta-1) z_i}\beta^{z_i} $$ and the corresponding posterior is $$ \pi(\beta,\pi|\mathbf{x},\mathbf{z}) \propto \beta^{-.999}\exp(-0.001\beta)\,\prod_{i=1}^n \pi^{z_i}(1-\pi)^{1-z_i}\,(1-x_i)^{(\beta-1) z_i}\beta^{z_i}, $$ where $\mathbf{z}=(z_1,\ldots,z_n)$.

As I dislike very much the BUGS type priors with almost zero parameters, I will now use $\beta^{-1}$ instead of $\beta^{-.999}\exp(-0.001\beta)$

Using this completion, the joint posterior distribution of $(\beta,\pi)$ can be written in closed form as $$ f(\beta,\pi|\mathbf{x},\mathbf{z}) \propto \pi^{\sum_{i=1}^nz_i}(1-\pi)^{n-\sum_{i=1}^nz_i}\,\beta^{\sum_{i=1}^n z_i-1}\left[\prod\nolimits_{z_i=1}(1-x_i)\right]^{\beta-1}, $$ If $\pi$ and $\beta$ are independent a priori, then, given $\mathbf{z}$, the vectors $\pi$ and $\mathbf{x}$ are independent; that is, $f(\pi|\mathbf{x},\mathbf{z})=f(\pi|\mathbf{z})$. Moreover, in that case, $\beta$ is also independent a posteriori from $\pi$ given $\mathbf{x}$ and $\mathbf{z}$, with density $f(\beta|\mathbf{x},\mathbf{z})$. If we apply the Gibbs sampler in this problem, it involves the successive simulation of $\mathbf{z}$ and $(\pi,\beta)$ conditional on one another and on the data:

Initialization: Choose $\pi^{(0)}$ and $\beta^{(0)}$ arbitrarily.

Iteration $t$ $(t\ge 1)$:

  1. For $i=1,\ldots,n$, generate $z_i^{(t)}$ such that $$\mathbb{P}\left(z_i=1\right)\propto \pi^{(t-1)}f\left(x_i|\beta^{(t-1)},z_i\right)$$
  2. Generate $\pi^{(t)}$ according to $f(\pi|\mathbf{z}^{(t)})$.
  3. Generate $\beta^{(t)}$ according to $f(\beta|\mathbf{z}^{(t)},\mathbf{x})$.

Try to solve the full conditionals before reading further

In your specific case, you can derive the full conditional $f(\pi|\mathbf{z})$ and $f(\beta|\mathbf{z},\mathbf{x})$ from the joint above: $$\eqalign{ f(\beta,\pi|\mathbf{x},\mathbf{z}) &\propto \pi^{\sum_{i=1}^nz_i}(1-\pi)^{n-\sum_{i=1}^nz_i}\,\beta^{\sum_{i=1}^n z_i-1}\left[\prod\nolimits_{z_i=1}(1-x_i)\right]^{\beta-1}\\ &\propto f(\pi|\mathbf{z}) f(\beta|\mathbf{x},\mathbf{z})\\ }$$

If you separate the terms in $\pi$ and the terms in $\beta$ you get $$f(\pi|\mathbf{z})\propto\pi^{\sum_{i=1}^n z_i}(1-\pi)^{n-\sum_{i=1}^nz_i}$$ which corresponds to a Beta $$\mathcal{B}e\left(1+\sum\nolimits_{i=1}^n z_i,1+n-\sum\nolimits_{i=1}^n z_i\right)$$ distribution and $$f(\beta|\mathbf{x},\mathbf{z})\propto\beta^{\sum_{i=1}^n z_i-1}\left[\prod\nolimits_{z_i=1}(1-x_i)\right]^{\beta}= \beta^{\sum_{i=1}^n z_i-1} \exp\left[-\beta\sum\nolimits_{z_i=1} \log(1-x_i)\right] $$ which corresponds to a Gamma $$\mathcal{Ga}\left(\sum\nolimits_{i=1}^n z_i,\sum\nolimits_{z_i=1} \log(1-x_i)\right)$$ distribution.

$\endgroup$
  • $\begingroup$ Thank you for the detailed explanation, I am working through the details. $\endgroup$ – user98056 Dec 14 '15 at 5:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.