# Questions tagged [gibbs]

The Gibbs sampler is a simple form of Markov Chain Monte Carlo simulation, widely used in Bayesian statistics, based on sampling from full conditional distributions for each variable or group of variables. The name comes from the method being first used on Gibbs random fields modeling of images by Geman and Geman (1984).

308 questions
302 views

### Why does Slice sampler use the log of the density?

The Slice sampler1 takes as its argument the log of the density to be sampled from. Why is it doing this? A commenter on this question pointed out that it makes no sense to "sample" from the log of ...
2k views

### Metropolis-Hastings using log of the density

Does Metropolis-Hastings work with the log of the proposal and the density to be sampled from? That is, say we want to sample from a density $\pi(x)$, using a proposal $q(x|x^{old})$, will the ...
3k views

### Clustering methods for unknown number of clusters

Matrix $X=[x_1,...,x_i,...,x_N]$ is a data-set containing $N$ data-points that each data-point $x_i$ is a vector of $D$ dimensions. Each dimension is a feature. The number of clusters ($K$) is unknown....
614 views

1k views

### When should we use Gibbs Sampling in a deep belief network? Before or after fine-tuning?

Gibbs sampling allows for sampling a vector with a deep belief network. There are two steps to training a DBN for a supervised learning task: greedy unsupervised pre-training and supervised fine-...
2k views

### How to derive Gibbs sampling?

I'm actually hesitating to ask this, because I'm afraid I will be referred to other questions or Wikipedia on Gibbs sampling, but I don't have the feeling that they describe what's at hand. Given a ...
1k views

### Marginal Likelihood from the Gibbs Output

I'm reproducing from scratch the results in Section 4.2.1 of Marginal Likelihood from the Gibbs Output Siddhartha Chib Journal of the American Statistical Association, Vol. 90, No. 432. (Dec., 1995)...
16 views

### What sort of data would be appropriate to analyze under an MCMC method?

MCMC methods describe stochastic sampling but I'm not entirely sure the contexts in real datasets one would wish to apply MCMC methods. What kind of data could I gain insight into with MCMC methods?
400 views

### Need help deriving a gibbs sampler for a normal mixture model with two components

Let $\theta_i$ be an indicator that the i-th eruption is a long eruption. (i.e. $\theta_i = 1$ if the i-th eruption is long and $\theta_i = 0$ otherwise.) Assume the following model and derive a Gibbs ...
70 views

### Sampling from a portion of the normal distribution?

I have a a conditional distribution $p(X_1 | \theta) \propto MVN(\mu, \Omega) \pi(X_1)$ where $X_1=[x_1, x_2, \dots, x_n]'$ and $\pi(X_1)=1$ when all $x_i \in [0,a)$ and $0$ otherwise. Is there any ...
273 views

### Gibbs Sampling and Probability Notation

Problem 1 I am trying to implement Gibbs Sampling for the following problem: There is a grid measuring 3 x 3 sites, each "site" can be designated in a state, $X$, of 1 or -1. The sites are numbered ...
106 views

### Gibbs within Metropolis

Consider a model with two parameters, $\alpha$ and $\beta$. We want to sample these two parameters conditioning on two data points, $d_1$ and $d_2$. Is it possible to use an algorithm like this: 1) ...
911 views

### Methods of fitting a dynamic linear model

I'm taking a time series course and am learning about exchangeable time series form of dynamic linear models (DLMs). This is given by: \begin{align*} \mathbf{y}_t' &= \mathbf{F}_t'\boldsymbol{\...
3k views

### Does the Gibbs Sampling algorithm guarantee detailed balance?

I have it on supreme authority1 that Gibbs Sampling is a special case of the Metropolis-Hastings algorithm for Markov Chain Monte Carlo sampling. The MH algorithm always gives a transition probability ...
728 views

### Is my OpenBUGS / WinBUGS model well specified?

I've just started trying to use OpenBUGS for Bayesian analysis of stochastic volatility models. In particular, I'm trying to calculate stochastic covariance, similar to the DC-MSV model specified by ...
538 views

134 views

### Understanding a measure of convergence of MCMC simulations

I am trying to better understand better the Gelman/Rubin measure of convergence of MCMCs. The method starts off by defining two quantities: $B$ and $W$. $B$ is said to be the between chain variance (...
885 views

### Is Slice Sampling a special case of Gibbs Sampling?

I read on this thread the following: If you can use both the gibbs sampler and slice sampling to sample from a posterior I would use the Gibbs sampler as the slice sampler seems unnecessary to ...
580 views

### Gibbs Sampling Detecting Change point in time series

I was reading through this one page paper on using Gibbs sampling for detecting a change point in a time series like data. While I understand the part where the $\lambda$ and $\phi$ are chosen from a ...
168 views

84 views

481 views

### Gibbs sampling for correlated random variables

Short summary Suppose two latent variables of a hierarchical model are correlated. Let $1-\epsilon$ be the degree of correlation. As $\epsilon\rightarrow 0$ the variables become perfectly correlated ...
8k views

### Gibbs sampling versus general MH-MCMC

I have just been doing some reading on Gibbs sampling and Metropolis Hastings algorithm and have a couple of questions. As I understand it, in the case of Gibbs sampling, if we have a large ...
30 views

### What methodology should I choose? If hiearchical, what model design?

I am working on a problem that has can beyond my level of understanding. I am quite familiar with R, so that would be my preferred choice but I also have access to SAS. Data I have created a fake ...
401 views

### Practical Implementation of Gibbs Sampling in Latent Dirichlet Allocation

In the collapsed Gibbs sampling version of LDA, the posterior distribution of topic assignments for each word is sampled. From what I have read (e.g. http://people.cs.umass.edu/~wallach/courses/s11/...
346 views

### Kalman filter conceptual question

I'm using the function dlmGibbsDIG (Gibbs sampler) in the dlmpackage from R to estimate the unknown variances. The output are ...
710 views

### Can I subsample a large dataset at every MCMC iteration?

Problem: I want to perform a Gibbs sampling to infer some posterior over a large dataset. Unfortunatelly, my model is not very simple and thus sampling is too slow. I would consider variational or ...
83 views

### What will happen if we sample the most probable value in the Gibbs sampling?

I am now working with the Gibbs sampling. One problem that puzzled me is that when we use the Gibbs sampling, we always sample randomly from the conditional probability. What will happen if we sample ...
365 views

### Gibbs sampling with Log-Normal observations

I am writing a Gibbs sampler for data that is Log-Normal (LN) distributed, with unknown mean and variance. There is a wealth of information on inference for LN models when either the mean or variance (...
3k views

### Gibbs sampling for Ising model

Homework question: Consider the 1-d Ising model. Let $x = (x_1,...x_d)$. $x_i$ is either -1 or +1 $\pi(x) \propto e^{\sum_{i=1}^{39}x_ix_{i+1}}$ Design a gibbs sampling algorithm to generate ...
5k views

### Gibbs sampling how to sample from the conditional probability? Bayesian model

I want to learn Gibbs sampling for a Bayesian model. How can I sample the variable from the conditional distribution? In this example, arrow means dependent; for example, ...
183 views

### Proper likelihood function in acceptance probability of Gibbs Sampler

I have a question about the acceptance ratio used when implementing a random walk M-H in a gibbs sampler to generate sample paths of an unobservable process. When computing the likelihood of a set of ...
504 views

### Gibbs sampling with Dirichlet Likelihood

I have a sequence of observations that I am representing as proportions: X1 X2 X3 X4 X5 0.10 0.20 0.50 0.12 0.08 0.07 0.24 0.55 0.04 0.10 ... ...
74 views

### EM on product of multinomials

I have the following conditional density: $$P(x | \theta, \pi) = \prod_{i=1}^I \prod_{j=1}^J t_{ij}! \prod_{k=1}^K \frac{1}{x_{ijk}!}(\sum_{l=1}^L \theta_{il} \pi_{jkl})^{x_{ijk}}$$ Here, $x$ is ...
1k views

### Gibbs measure and normal distribution

On Wikipedia, the Gibbs measure defines the probability as: $$P(X=x) = \frac{1}{Z(\beta)}\exp(-\beta E(x))$$ Now, the familiar form of the normal distribution is:  P(x) = \frac{1}{\sqrt{2\pi}\...
468 views

### Gibbs sampler for conditionals that are exponential: Example from Casella & George paper

I am trying to work out Example 2 from Casella and George's paper "Explaining the Gibbs Sampler" in R. The example is: ...
1k views

### Gibbs sampling for LDA — does a small Dirichlet concentration parameter make a difference?

I'm using a Gibbs sampler for Latent Dirichlet allocation as described by Griffiths and Steyvers (http://www.ncbi.nlm.nih.gov/pmc/articles/PMC387300/). The sampling of a new topic $j$ for word $i$ is ...
I am trying to estimate the precision $\tau$ of a normal distribution with either WinBUGS or OpenBUGS: $c \sim \text{normal}(\mu,\tau)$ $\mu \rightarrow \lambda \cdot t^{-\beta}$ \$\tau \sim \text{...