# Tag Info

Accepted

### When would one use Gibbs sampling instead of Metropolis-Hastings?

Firstly, let me note [somewhat pedantically] that There are several different kinds of MCMC algorithms: Metropolis-Hastings, Gibbs, importance/rejection sampling (related). importance and ...
• 106k

• 4,236
Accepted

### Is the joint distribution $P_{XY}(x,y)$ determined from the conditionals $P_{X|Y}(x|y)$ and $P_{Y|X}(y|x)$?

This characterisation of the joint by the conditionals is the Hammersley-Clifford theorem. Unpublished by the authors but later established under the positivity condition by Julian Besag in 1974. The ...
• 106k

### Rao-Blackwellization of Gibbs Sampler

The Gibbs sampler can then be used to improve efficiency of (say) samples from a marginal posterior, call it $\pi_2(\theta_2|y)$. Note \begin{eqnarray*} \pi_2(\theta_2|y)&=&\int \pi(\theta_1,\...
• 33.6k
Accepted

### Sampling from an Inverse Gamma distribution

This discrepancy arises because there are two different parameterizations of the Gamma distribution and each relate differently to the Inverse Gamma distribution. On Wikipedia, the two ...
• 15.6k

• 139k

• 106k
Accepted

### Reversibility in MCMC

I am interpreting your question as much more general in that "Is there any gain to using a reversible Markov chain over a non-reversible Markov chain?". Here are two reasons I can think of off the top ...
• 15.6k

### Metropolis-Within-Gibbs sampling with only marginal distribution known for a subset of variables

In an ideal world, sampling from $p_1(x_1)$ and then from $p_{1|2}(x_2|x_1)$ is a correct way to simulate from the joint. In case one of these distributions is unavailable, simulating a single step of ...
• 106k
Accepted

### What is the difference between monte carlo integration and gibbs sampling?

Monte Carlo integration is a technique for numerically integrating a function by evaluating it at many randomly chosen points. It's useful for computing integrals when a closed form solution doesn't ...
• 32.7k

### Conditional distribution for Gibbs sampling for Gaussian mixture

The Gibbs steps for a mixture model are to be found in all papers and books addressing Bayesian inference on mixtures, from our early paper with Diebolt (1990) to the reference book of Sylvia ...
• 106k

### Gibbs algorithm using negative binomial produces NAs

In short, the problem behind the question is statistically meaningless if probabilistically interesting. When using the definition of the Negative Binomial random variate as the number of failures, ...
• 106k
Accepted

### Monte Carlo Options for Data Augmentation

As noted p. 530 in the Tanner & Wong (1987) paper, the method applies to $$p_i(\theta|y) = m^{-1}\sum_{j=1}^m p(\theta|z^{(j)},y)$$ even when $m=1$. In this special case, the algorithm is a Gibbs ...
• 106k
Accepted

### How to sample using Gibbs with a uniform latent variable?

If $u_i$ has to be between $e^{-z}$ and $1$, then $e^{-z}$ needs to be between $0$ and $u_i$. Thus the distribution of $z$ given $\mathbf u$ is effectively a standard exponential RV with truncation: \...
• 8,172
Accepted

### How does Gibbs sampling produce values for a variable using the univariate conditional probability?

You raised two broad questions How expensive is Gibbs sampling? In your multivariate Gaussian example, the domains don't seem to match up, since the values obtained via Gibbs sampling seem to be ...
• 15.6k
Accepted

### Gibbs sampling and Conjugate Priors

You don't require conjugate priors for Gibbs sampling. What you need to be able to do is produce samples from the full conditional distributions. Conjugate priors generally make that easier, but ...
• 284k
Accepted

### Sampling a random binary matrix with "Gaussian" probability distribution

The target probability mass function is of the form \begin{align}p(A)&\propto \exp\left\{-\frac 12 \mathrm{tr}\left[\left(A-M\right)^TV\left(A-M\right)\right]\right\}\\ &\propto \exp\left\{-\...
• 106k

### The Harris recurrence of a stepping-out slice-sampling-within-Gibbs MCMC

I prove that the stepping-out and shrinkage procedure satisfies detailed balance, but this is of course not enough to show irreducibility or ergodicity. And it's easy to construct examples in which ...
Accepted

### handling metropolis hastings rejection during a Gibbs sweep

As you state, the "first part uses Metropolis Hastings to find the next parameter value" this means the exact simulation from the posterior is replaced with one (or several) Metropolis Hastings ...
• 106k

### Is Coordinate Ascent algorithm related to Gibbs Sampling in some way?

The purpose of coordinate ascent is to maximize a function, whereas the purpose of Gibbs sampling is to draw samples from a probability distribution. The two methods are loosely related in the sense ...
• 32.7k