# Questions tagged [gibbs]

The Gibbs sampler is a simple form of Markov Chain Monte Carlo simulation, widely used in Bayesian statistics, based on sampling from full conditional distributions for each variable or group of variables. The name comes from the method being first used on Gibbs random fields modeling of images by Geman and Geman (1984).

340 questions
Filter by
Sorted by
Tagged with
58k views

### Can someone explain Gibbs sampling in very simple words? [duplicate]

I'm doing some reading on topic modeling (with Latent Dirichlet Allocation) which makes use of Gibbs sampling. As a newbie in statistics―well, I know things like binomials, multinomials, priors, etc.―,...
7k views

### OpenBugs vs. JAGS

I am about to try out a BUGS style environment for estimating Bayesian models. Are there any important advantages to consider in choosing between OpenBugs or JAGS? Is one likely to replace the other ...
13k views

### What is the difference between Metropolis-Hastings, Gibbs, Importance, and Rejection sampling?

I have been trying to learn MCMC methods and have come across Metropolis-Hastings, Gibbs, Importance, and Rejection sampling. While some of these differences are obvious, i.e., how Gibbs is a special ...
11k views

### A good Gibbs sampling tutorials and references

I want to learn how Gibbs Sampling works and I am looking for a good basic to intermediate paper. I have a computer science background and basic statistic knowledge. Anyone has read good material ...
1k views

### What are some well known improvements over textbook MCMC algorithms that people use for bayesian inference?

When I'm coding a Monte Carlo simulation for some problem, and the model is simple enough, I use a very basic textbook Gibbs sampling. When it's not possible to use Gibbs sampling, I code the textbook ...
9k views

### Gibbs sampling versus general MH-MCMC

I have just been doing some reading on Gibbs sampling and Metropolis Hastings algorithm and have a couple of questions. As I understand it, in the case of Gibbs sampling, if we have a large ...
10k views

### When would one use Gibbs sampling instead of Metropolis-Hastings?

There are different kinds of MCMC algorithms: Metropolis-Hastings Gibbs Importance/rejection sampling (related). Why would one use Gibbs sampling instead of Metropolis-Hastings? I suspect there ...
3k views

### Does the Gibbs Sampling algorithm guarantee detailed balance?

I have it on supreme authority1 that Gibbs Sampling is a special case of the Metropolis-Hastings algorithm for Markov Chain Monte Carlo sampling. The MH algorithm always gives a transition probability ...
2k views

### Stan $\hat{R}$ versus Gelman-Rubin $\hat{R}$ definition

I was going through the Stan documentation which can be downloaded from here. I was particularly interested in their implementation of the Gelman-Rubin diagnostic. The original paper Gelman & ...
7k views

### Where do the full conditionals come from in Gibbs sampling?

MCMC algorithms like Metropolis-Hastings and Gibbs sampling are ways of sampling from the joint posterior distributions. I think I understand and can implement metropolis-hasting pretty easily--you ...
2k views

### Marginal Likelihood from the Gibbs Output

I'm reproducing from scratch the results in Section 4.2.1 of Marginal Likelihood from the Gibbs Output Siddhartha Chib Journal of the American Statistical Association, Vol. 90, No. 432. (Dec., 1995)...
327 views

### Why does the redundant mean parameterization speed up Gibbs MCMC?

In Gelman & Hill (2007)'s book (Data Analysis Using Regression and Multilevel/Hierarchical Models), the authors claim that including redundant mean parameters can help speed up MCMC. The given ...
872 views

### Is Gibbs sampling an MCMC method?

As far as I understand it, it is (at least, that is how Wikipedia defines it). But I've found this statement by Efron* (emphasis added): Markov chain Monte Carlo (MCMC) is the great success story ...
3k views

### Gibbs sampling for Ising model

Homework question: Consider the 1-d Ising model. Let $x = (x_1,...x_d)$. $x_i$ is either -1 or +1 $\pi(x) \propto e^{\sum_{i=1}^{39}x_ix_{i+1}}$ Design a gibbs sampling algorithm to generate ...
3k views

### How to derive Gibbs sampling?

I'm actually hesitating to ask this, because I'm afraid I will be referred to other questions or Wikipedia on Gibbs sampling, but I don't have the feeling that they describe what's at hand. Given a ...
1k views

### How do programs like BUGS/JAGS automatically determine conditional distributions for Gibbs sampling?

Seems like full conditionals are often quite difficult to derive, yet programs like JAGS and BUGS derive them automatically. Can someone explain how they algorithmically generate full conditionals for ...
631 views

188 views

### Would an “importance Gibbs” sampling method work?

I suspect this is a fairly unusual and exploratory question, so please bear with me. I am wondering if one could apply the idea of importance sampling to Gibbs sampling. Here's what I mean: in Gibbs ...
3k views

### Bayesian estimation of Dirichlet distribution parameters

I want to estimate parameters of Dirichlet mixture models using Gibbs sampling and I have some questions about that: Is a mixture of Dirichlet distributions equivalent to a Dirichlet process? What is ...
I am confused about the correct way to write the elastic net. After reading some research papers there seems to be three forms 1) $\exp\{-\lambda_1|\beta_k|-\lambda_2\beta_k^2\}$ 2) \exp\{-\frac{(\... 1answer 742 views ### Can I subsample a large dataset at every MCMC iteration? Problem: I want to perform a Gibbs sampling to infer some posterior over a large dataset. Unfortunatelly, my model is not very simple and thus sampling is too slow. I would consider variational or ... 1answer 14k views ### Gibbs sampler examples in R [closed] How can I implement Gibbs sampler for the posterior distribution, and estimating the marginal posterior distribution by making histogram? 2answers 8k views ### Generating samples from Gibbs sampling I am quite new to sampling. I am doing Gibbs sampling for a Bayesian network. I am aware about the algorithm for the Gibbs sampling but there's one thing I am not able to understand. For example let'... 2answers 2k views ### Gibbs sampler from conditional distribution I am trying to propose Gibbs sampling with the density below, $$p(y_1,y_2,y_3)\propto \exp [-({{y}_{1}}+{{y}_{2}}+{{y}_{3}}+{{\theta}_{12}{y_1}{y_2}}+{{\theta }_{13}{y_3}{y_1}}+{{\theta }_{23}{y_2}{... 2answers 6k views ### Gibbs sampling how to sample from the conditional probability? Bayesian model I want to learn Gibbs sampling for a Bayesian model. How can I sample the variable from the conditional distribution? In this example, arrow means dependent; for example, ... 3answers 2k views ### Why does sampling from the posterior predictive distribution p(x_{new} \mid x_1, \ldots x_n) work without having to average out the integral? In a Bayesian model, the posterior predictive distribution is usually written as:$$ p(x_{new} \mid x_1, \ldots x_n) = \int_{-\infty}^{\infty} p(x_{new}\mid \mu) \ p(\mu \mid x_1, \ldots x_n)d\mu $$... 1answer 4k views ### Gibbs sampling to produce posterior pdf Suppose we have the following classical normal linear regression model:$$y_i = \beta_1 x_{1i} + \beta_2x_{2i} + \beta_3x_{3i} + e_i$$where e_{i} \sim iid.N(0, \sigma^2) for all i = 1, 2, \cdots,... 1answer 198 views ### Gibbs Sampler output: how many Markov chains? When running a Gibbs sampler (for n=200 Iterations) with two full conditionals, I get the output \mathbf{x} = (x_1^{(n)},x_2^{(n)})_{n =1,...,200}. So \mathbf{x} is the realizations of a Gibbs ... 1answer 227 views ### Given that one can sample X \sim f(x), is there an easy way to sample Y \sim k \cdot f(g(y)) (such as k \cdot f(e^y))? Say I'm able to sample an RV X from a PDF f(x), can I exploit this to efficiently sample another RV Y \sim k \cdot f(g(y)) (where k is a normalizing constant)? I'm interested in something ... 3answers 258 views ### Conditional distribution of \exp(-|x|-|y|-a \cdot |x-y|) I am trying to use Gibbs sampling or Metropolis-Hastings to draw samples from the joint distribution$$f(x,y)\propto\exp(-|x|-|y|-a \cdot |x-y|)For this I need the conditional distributions of x ... 2answers 1k views ### Gibbs sampler gets stuck in local mode I am very new to statistics and trying to implement a Gibbs sampler. However, according to wikipedia https://en.wikipedia.org/wiki/Gibbs_sampling and this discussion thread http://metaoptimize.com/qa/... 1answer 1k views ### Zero-inflated Poisson and Gibbs sampling, proofs and sampling I am trying to figure out zip-inflated Poisson (ZIP) model. In this model, random data X_1, .., X_n are of the form X_i=R_iY_i, where the Y_i's have Poisson distribution (\lambda) and the R_i... 1answer 2k views ### Metropolis-Within-Gibbs sampling with only marginal distribution known for a subset of variables Typically in Gibbs sampling we want to sample from a joint distribution p(X_1, X_2, ..., X_N), but because the joint is hard to sample from directly, we instead achieve this by iteratively sampling ... 2answers 911 views ### slice sampling within a Gibbs sampler Questions My questions are: Is the following slice-sampling-within-Gibbs approach valid? Is there a good reference out there that uses, or better yet, justifies it? Context I'm trying to sample ... 1answer 303 views ### Gibbs Sampler contradiction proof I want to prove that the systematic scan Gibbs sampler yields an aperiodic chain X on a general state space. Let \pi be the stationary distribution for the resulting chain. Suppose to get a ... 1answer 1k views ### Methods of fitting a dynamic linear model I'm taking a time series course and am learning about exchangeable time series form of dynamic linear models (DLMs). This is given by: \begin{align*} \mathbf{y}_t' &= \mathbf{F}_t'\boldsymbol{\... 1answer 203 views ### Dirichlet process mixture MCMC I'm reading Markov Chain Sampling Methods for Dirichlet Process Mixture Models by Radford M. Neal. Equation (3.6) states that \text{If } c=c_{j} \text{ for some } j\neq i: P\left(c_{i}=c\;|\;c_{-i}... 0answers 805 views ### Is my OpenBUGS / WinBUGS model well specified? I've just started trying to use OpenBUGS for Bayesian analysis of stochastic volatility models. In particular, I'm trying to calculate stochastic covariance, similar to the DC-MSV model specified by ... 2answers 3k views ### How to find conditional distributions from joint I want to learn about how to do Gibbs sampling, starting with finding conditional distributions given a joint distribution. While looking for examples, I found this blog post that I wanted to ... 2answers 2k views ### MCMC chain getting stuck I am trying to use a Metropolis-within-Gibbs type algorithm to sample\theta$and$x$from the following model. Starting with Bayes theorem I can write:$$P(\theta, x | y) = \frac{P(y | x, \theta) ... 1answer 3k views ### Sampling from an Inverse Gamma distribution I am using Gibbs sampling in the MCMC estimation of a stochastic volatility model. One of the posterior distributions is an Inverse Gamma distribution.I was struggling with the sampling procedure or ... 1answer 2k views ### Preparing Bayesian conditional distributions for Gibbs sampling I was looking at the Gibbs Sampler when I stumbled upon the following example: Suppose$y = (y_{1}, y_{2}, \ldots, y_{n})$are iid observations from an$N(\mu, \tau^{-1})$Furthermore, suppose there ... 1answer 2k views ### Bayesian regression full conditional distribution I have a problem with the derivation of the full conditional distribution of the regression coefficients in a simple Bayesian regression. The source of the following equations is: Lynch (2007). ... 1answer 960 views ### Gibbs sampling from a complex full conditional I have a sampling question relating to Gibbs sampling of a complicated full conditional. Supposed I have a complicated full conditional that I want a single sample from$p(\theta_i$|$\theta_{-i}$,$...
I am self-studying Gibbs sampling from a book. The book introduces metropolis hastings algortihm to generate representative values from a posterior distribution. So we know $p(D | \theta) p(\theta)$ ...