# Questions tagged [gibbs]

The Gibbs sampler is a simple form of Markov Chain Monte Carlo simulation, widely used in Bayesian statistics, based on sampling from full conditional distributions for each variable or group of variables. The name comes from the method being first used on Gibbs random fields modeling of images by Geman and Geman (1984).

317 questions
Filter by
Sorted by
Tagged with
502 views

### Prior of multivariate Polya distribution?

Anyone knows a prior (preferably conjugate) to the multivariate Polya distribution? I need it for Gibbs sampling. So if anyone has another idea, I am interested.
36 views

82 views

### Sampling posterior of empty cluster in GMM and Gibbs

Consider performing inference via a standard Gibbs sampler for a standard Gaussian Mixture Model (GMM) with $k$ components that are Gaussians $$\mathcal{N}(\mu_{k}, \sigma^{2}_{k})$$ where we assume ...
483 views

### Blocked Gibbs Sampling using Forward / Backward Algorithm

I am new to machine learning and have been reading about gibbs sampling. From my understanding, a Gibbs algorithm samples a single variable iteratively conditioned on all other variables. In blocked ...
284 views

### Gibbs Sampler - Sample mean convergence

To simulate from the posterior distribution $p(\theta|Y)$ where $\theta = (\mu,\lambda_1,\lambda_2)$, I run a Gibbs sampler to draw approximately random values from $p(\theta|Y)$. This Gibbs sampler ...
402 views

### Need help deriving a gibbs sampler for a normal mixture model with two components

Let $\theta_i$ be an indicator that the i-th eruption is a long eruption. (i.e. $\theta_i = 1$ if the i-th eruption is long and $\theta_i = 0$ otherwise.) Assume the following model and derive a Gibbs ...
96 views

34 views

### antagonistic simulated annealing

Simulated annealing aims at a series of target distributions $$\pi_T(x)\propto\exp\{T\,H(x)\}$$ to find the maximum of the function $H$ and its argument $$\arg_x\max_{x\in \mathfrak X} H(x)$$ if the ...
80 views

### Gibbs Sampling vs. Using Raw Probability in Contrastive Divergence

In Hinton's Practical Guide to Training Restricted Boltzmann Machines, Section 3, he discusses different situations in which one should take a sample from the Gibbs sampling process, and other ...
193 views

### Is burn-in necessary for MCMC/Gibbs sampling if I have samples from the true distribution already?

Say I have some samples from a distribution $p$, and I want to get more samples using MCMC/Gibbs sampling. Since the existing samples are known from the equilibrium distribution $p$, if I use them as ...
277 views

### Convergence of approximate Gibbs sampling

We have a bivariate random variable $(X,Y)$ for which sampling is challenging. If we were to know how to sample from the conditionals $(X|Y)$ and $(Y|X)$, we could get samples from the joint using ...
252 views

### Gibbs sampling for parameter estimation

I am reading the paper by Willemsen et al (2015), "A multivariate Bayesian model for embryonic growth", Statistics in Medicine, 34:8, 1351–1365 where they define the posterior distribution as, \begin{...
711 views

### Bayesian estimation of Dynamic Regression with AR(1) parameters

I would like to draw (Bayesian) inference in a dynamic linear regression with regression parameters following independent AR(1) processes $\beta_{t,i} = \mu_i+\beta_{t-1,i}+w_{t,i}$. However, I ...
292 views

### When Gibbs Sampling is fast/slow to converge?

Are there any heuristic/theories showing that on what kind of Bayesian models the convergence of Gibbs sampling is fast/slow? For example, from my limited experience, I feel (may be wrong) when a ...
589 views

### Criteria in determining “step size” of Metropolis-hasting algorithms

I am training a complex Bayesian model using Gibbs sampling and Metropolis-Hasting algorithm. Most of the parameters are directly sampled by using conjugate priors except for 3 params which are ...
124 views

### When 2% of the Bayesian Model have not converged?

I have model with 20000 latent parameters, set up in a Gibb's sampler. 98% of the parameters and sometimes 99.5% of the parameters satisfy the Geweke convergence statistic, have low autocorrelation ...
171 views

### Derive Marginal Posterior to set up Gibbs-Sampler

I am currently trying to replicate a Hierarchical Model for multivariate returns proposed in the paper Portfolio selection using hierarchical Bayesian analysis and MCMC methods. However, in order to ...
593 views

### Efficiency in Metropolis Vs Gibbs sampling

I have read that Gibbs sampling is more efficient than Metropolis algorithm. Why? Is this due only to the fact the in Gibbs sampling the acceptance rate is $1$, so that the chain needs fewer ...
2k views

2k views

### Metropolis-Hastings Algorithm within Gibbs Sampling

I have this $f$ function below. $$f(x_1,x_2)\propto \left(\dfrac{x_1}{x_2}\right)\left(\dfrac{\alpha}{x_2}\right)^{x_1-1}exp\left\{-\left(\dfrac{\alpha}{x_2}\right)^{x_1} \right\}I_{R^+}(x)$$ where ...
376 views

### Gibbs sampler implementation

I am just getting started with the Gibbs Sampler and came across an implementation from here and here and here. All of theses implementations are based on the first article. There is an inner loop ...
376 views

### Gibbs sampling on the use of gamma distribution

I am trying to reproduce a Gibbs sampling but some points are unclear to me. Let's assume a linear model of the following form $$y=x\beta + \epsilon$$ The prior distribution for the beta ...
2k views

### How does Gibbs sampling produce values for a variable using the univariate conditional probability?

I have a question about Gibbs sampling for generating samples. The Gibbs sampling algorithm is often stated. $x^0 = (x_1^0, x_2^0, \ldots, x_n^0)$ //initialize random values for $t=1$ in $T$ //...
167 views

### Why can Gibbs sampling outputs be used in Rao-Blackwellization?

I'm currently learning Chib (1995)'s method to calculate the marginal likelihood of a Bayesian model using Gibbs sampling outputs. I'm stuck in the Rao-Blackwellization step. Suppose $\mu$ and $\phi$...
208 views

### Gibbs sampling from conditional full posterior distribution

I am reading the paper by Willemsen et al (2015), "A multivariate Bayesian model for embryonic growth", Statistics in Medicine, 34:8, 1351–1365 where they define the posterior distribution as, \begin{...
293 views

### Conditional distribution for Gibbs sampling for Gaussian mixture

If we draw $n$ i.i.d. points $x_1,x_2,\dots,x_n$ from the following Gaussian mixture: $$\frac 12 \mathcal N(x \mid \mu_1,1) + \frac 12 \mathcal N(x\mid \mu_2,1)$$ and the prior $p(\mu_1 , \mu_2 )$ ...
147 views

### Is Coordinate Ascent algorithm related to Gibbs Sampling in some way?

I wonder if only me feel there are certain connections between them, I googled it for a long time, but found no where mentioned these two method. But to me, they indeed looked so related, Could anyone ...
980 views

### Implementing Gibbs sampler in R from posterior distribution

I am referencing a follow-up idea from something I posted earlier (Zero-inflated Poisson and Gibbs sampling, proofs and sampling). I want to implement the Gibbs sampler, by generating a large (...
2k views

### Gibbs sampling from posterior distribution using R

New to MCMC. I have a model, saying $$Y_i=\beta_0+\beta_1x_{i1}+\beta_2x_{i2}+\frac{e_i}{\sqrt{\mu}}$$ where $x_{ij}$ are fixed covariates, $e_i\sim N(0,1)$, $\beta_0$, $\beta_1$, $\beta_2$ and $\mu$ ...
233 views

### Drawing from a conditional density

I have a simple question. Suppose $X=(X_1,X_2,X_3)$ is multivariate normal. What's the best (quickest) way to draw from the conditional density $X_1\mid \exp(X_1)+\exp(X_2)+X_3$?
54 views

### What's the role of the data in Gibbs Sampling?

Trying to wrap my mind around Gibbs Sampling. Across many answers in this same forum, I constantly notice that the examples shown do not actually require an observed data set (First example (with R ...
440 views

### Gibbs sampling and mixed distribution

For a project, I need to simulate from a joint distribution with both continuous and discrete variables that are dependent. The conditional distribution of any variable given the rest is known. I ...
109 views

### Efficiently sampling from mixture distribution posterior

I have the following model:  \begin{align} \pi_1\sim & \text{Unif}(0,1)\\ \lambda_1,\lambda_2\sim & \text{Ga}(1,1)\\ z_i\sim & \pi_1^{1(z_i=1)}\pi_2^{1(z_i=2)}\\ p(y_i|\lambda_1,\...
80 views

### Why does $P(\theta_1\mid D, \theta_2) \propto P(D \mid \theta_1, \theta_2)P(\theta_1)$ hold?

Suppose that in a Bayesian framework we have observed data $D$, using independent prior distributions on the parameters of the model, denoted by $\theta_1, \theta_2$. Then, the joint posterior ...
348 views

197 views

### Does Gibbs sampling MCMC place limitations on the posterior?

I'm starting to learn about Gibbs sampling, having so far only worked with Metropolis-Hastings MCMC, and there's something I haven't grasped yet about the way Gibbs sampling works. To frame the ...
162 views

### MCMC using GIBBS sampling: can different burn-in be used for different parameters?

I have run a stochastic volatility model with 4 parameters. I have used the Heidelberg and Welch convergence diagnostic. The result shows 3 out of 4 parameters have passed the stationary and half-...
120 views

### How to estimate biases from coin and dice using only observed dice throws in this setup?

To help me understand some concepts I'm learning in my first exposition to machine learning, I'm trying to tackle the following "simple" problem The setup of the problem is as follows: My friend ...
183 views

### Derive the Gibbs sampler for this bivariate distribution

I understand the theory of Gibbs sampling. It is an iterative sampling algorithm that defines, sequence of random variables with the property of a Markov chain. Specifically, I choose any starting ...
1k views

### When should we use Gibbs Sampling in a deep belief network? Before or after fine-tuning?

Gibbs sampling allows for sampling a vector with a deep belief network. There are two steps to training a DBN for a supervised learning task: greedy unsupervised pre-training and supervised fine-...
I am trying to estimate the precision $\tau$ of a normal distribution with either WinBUGS or OpenBUGS: $c \sim \text{normal}(\mu,\tau)$ $\mu \rightarrow \lambda \cdot t^{-\beta}$ \$\tau \sim \text{...