# Questions tagged [gibbs]

The Gibbs sampler is a simple form of Markov Chain Monte Carlo simulation, widely used in Bayesian statistics, based on sampling from full conditional distributions for each variable or group of variables. The name comes from the method being first used on Gibbs random fields modeling of images by Geman and Geman (1984).

317 questions
Filter by
Sorted by
Tagged with
293 views

### Conditional distribution for Gibbs sampling for Gaussian mixture

If we draw $n$ i.i.d. points $x_1,x_2,\dots,x_n$ from the following Gaussian mixture: $$\frac 12 \mathcal N(x \mid \mu_1,1) + \frac 12 \mathcal N(x\mid \mu_2,1)$$ and the prior $p(\mu_1 , \mu_2 )$ ...
43 views

### Sampling from partial posterior

I'm reading a paper where the authors have something like the following as one step in their MCMC: \begin{align} y&=\rho_1 x_1+\rho_2 x_2+\epsilon\\ z&=\beta\tilde{y}+u \end{align} ...
2k views

### How to find conditional distributions from joint

I want to learn about how to do Gibbs sampling, starting with finding conditional distributions given a joint distribution. While looking for examples, I found this blog post that I wanted to ...
44 views

### HDP: Gibbs sampler implementation

I am trying to recreate the model proposed by Gao et al. (2011), based on the Hierarchical Dirichlet Process proposed by Teh and al. (2005). To estimate the model (let's call it iHDP) I need to ...
89 views

### Can Bayesian Optimization solve this problem?

Suppose ${\bf{x}} = (x_1,\ldots,x_n)$ and $f({\bf{x}})\propto 1_A({\bf{x}}) \prod_{i=1}^n {x_i}^{\alpha_i-1} e^{-\beta_i x_i}$ , i.e. $f$ is proportional to the product of independent gamma ...
614 views

### Any good book for learning probability programming

Are there any good books for me to learn probability programming? For example, I am new to Latent Dirichlet allocation (LDA) and Gibbs sampling. I have read some books about the techniques, but it ...
38 views

### Find joint maximum of a sampled density

I used a sampling method to fit a model with three parameters to data, by supplying the likelihood function and priors. (I'm using JAGS but I think this applies to any method). I obtain triplets of ...
440 views

### What is the difference between monte carlo integration and gibbs sampling?

I am aware that both are methods of sampling from the posterior. MC integration replaces the integral by a sample MC sample. Is this sample independent? Gibbs sampling is a class of MCMC ...
8k views

### When would one use Gibbs sampling instead of Metropolis-Hastings?

There are different kinds of MCMC algorithms: Metropolis-Hastings Gibbs Importance/rejection sampling (related). Why would one use Gibbs sampling instead of Metropolis-Hastings? I suspect there ...
50 views

### Approximately sampling $(X, Y)$ when sampling $X$ is easy

Suppose I am interested in sampling many pairs $(\mathbf X, Y)$ from some distribution $f(\mathbf x, y)$ where $\mathbf x \in \mathbb R^p$, $p$ large ; I am interested in both exact and approximate ...
394 views

### Gibbs Sampler Running Wild

So, I'm setting up a Gibbs Sampler using a multivariate normal model with a Jeffreys prior (working through the Hoff book on my own). There's also missing data to be imputed. I've checked my posterior ...
29 views

### Slice Sampling asks to draw from $f^{-1}]y,+\infty[$

Slice Sampling asks to draw uniformly from $f^{-1}]y,+\infty[$. Wikipedia page However, how can we be sure that a uniform defined over the set $f^{-1}]y,+\infty[$ is in fact proper? If I had to ...
486 views

### Can one use the RTS (Rauch–Tung–Striebel) smoother to simulate latent factors (as opposed to Carter and Kohn procedure)?

It is a little surprising that I have not found anything online in the literature which clarified this. I am working on an MCMC Gibbs sampling procedure to calibrate a "dynamic factor model". One of ...
790 views

### Beta distribution parameter estimation: method of moments

In a paper: Topics over time, method of moments was applied to estimate $\alpha$ and $\beta$ for a Beta distribution. My question is that how $\alpha$ or $\beta$ should be calculated if there are no ...
82 views

### Sampling posterior of empty cluster in GMM and Gibbs

Consider performing inference via a standard Gibbs sampler for a standard Gaussian Mixture Model (GMM) with $k$ components that are Gaussians $$\mathcal{N}(\mu_{k}, \sigma^{2}_{k})$$ where we assume ...
153 views

572 views

### deriving posterior conditionals for gibbs sampling

I'm new to Bayesian inference and Gibbs sampling in general, and I'm struggling trying to derive the conditional posteriors for a particular data generating process I'm trying to model. The model I ...
66 views

### Generating samples for $p(\theta_{i}|\pmb{x})$ if samples from $p(\phi|\pmb{x})$ are known

Suppose $X_{i}|\theta_{i} \sim D_{1}(\theta_{i})$ and $\theta_{i}|\phi \sim D_{2}(\phi)$. Moreover $\phi \sim D_{3}(c)$ where c is known. How would I generate samples for $p(\theta_{i}|\pmb{x})$ if I ...
277 views

### Convergence of approximate Gibbs sampling

We have a bivariate random variable $(X,Y)$ for which sampling is challenging. If we were to know how to sample from the conditionals $(X|Y)$ and $(Y|X)$, we could get samples from the joint using ...
I'm trying to understand how the Gibbs sampling algorithm works. I've simplified it into a bivariate case to help my understanding but I'm unsure how to go from conditioning on $X^{t-1},Y^{t-1},X^t$ ...