Questions tagged [gibbs]
The Gibbs sampler is a simple form of Markov Chain Monte Carlo simulation, widely used in Bayesian statistics, based on sampling from full conditional distributions for each variable or group of variables. The name comes from the method being first used on Gibbs random fields modeling of images by Geman and Geman (1984).
420
questions
0
votes
0
answers
9
views
Meaning and importance of 'Gibbs update' in MCMC
I am studying MCMC by "Handbook of Markov Chain Monte Carlo" by Brooks, Gelman
This book is nice to explaining many fundamental concepts regarding MCMC. Especially in first chapter, they ...
- 45
0
votes
1
answer
40
views
Block sampling hidden state using forward algorithm only
In a hidden Markov model, I can't get my mind around why I can't sample the full hidden state $\vec x$ using only a forward sampling algorithm.
Let $\vec y$ be the observed data and $\theta$ the model ...
- 53
0
votes
0
answers
56
views
Bayesian model (Gibbs sampling) with frequency data for non-standard distribution
I have the following model.
\begin{align}
X &= D\,(F+S\,(1+G)) \\
D &\sim \mathrm{Exp}\,(\lambda) \\
F &\sim \mathrm{Unif}\,(0,1) \\
S &\sim \mathrm{Poi}\,(\sigma) \\
G &\sim \...
- 143
0
votes
0
answers
10
views
When to stop MCMC within collapsed Gibbs?
I am setting up a Hierarchical model whose target distribution is $p(\theta,w|y)$, $\theta$ being a reduced set of high-level parameters, $w$ being a data augmentation of very high dimension, and $y$ ...
- 1
0
votes
0
answers
44
views
Random scan Gibbs sampling as special case of Metropolis-Hastings
I am reading Blitzstein's Introduction to Probability and come across with the following proof that I don't really understand:
Theorem: The random scan Gibbs sampler is a special case of the ...
0
votes
0
answers
12
views
Is it possible to increase the Hastings ratio by combining and mixing elementary kernels?
Let's say I am working with a state $X$ split into three parts $U$, $V$, and $W$.
I can efficiently sample from $W|U,V$, $U|V$, and $V|U$. My initial intuition was to do a variable-at-a-time ...
- 53
0
votes
0
answers
10
views
Estimating integrated-out variables
An illustration
David Blei in his lecture notes considers a collapsed Gibbs sampler for a Gaussian mixture model. In this case $B = (\mu_1, \dotsc, \mu_K)$ is a latent vector of means and $A = (Z_1, \...
- 143
1
vote
0
answers
15
views
Gibbs sampler, how to generate samples based on odds?
I am reading a paper that contains a Gibbs sampler for a regression model with parameters $\beta$ and design matrix $X$. One of the steps in the Gibbs sampler requires simulating from a binary random ...
- 11
0
votes
0
answers
19
views
Can I add more steps to a Gibbs sampler without hurting the ergodicity of the chain?
I have a Gibbs sampler that updates a system of $n$ variables $(x_1,\ldots,x_n)$ by each of the full conditional distributions. Let's say that I add a $n+1^{th}$ step: I also update $v^T(x_1,\ldots,...
- 1
1
vote
0
answers
30
views
Gibbs sampler: Conditional distribution with nested latent variable distributions
I have the following model (simplified here for the description). The obsvered variable is $y_i$ which is a linear function of some random variable $\eta_i$ and a random error term $\epsilon_i$:
$y_{i}...
- 155
4
votes
1
answer
364
views
Monte Carlo Options for Data Augmentation
In the seminal paper by Tanner and Wong (1987) on data augmentation, they describe a method for obtaining the posterior distribution $p(\theta|y)$ by data augmentation. Let $Z$ be a latent variable (...
- 193
0
votes
0
answers
30
views
Conditionally conjugate prior for non-nested (i.e. crossed) normal model?
I am trying to write/understand a conditionally-conjugate Gibbs sampler for what is essentially a linear, mixed effects model. I more or less get the conditionally-conjugate posterior for the ...
- 101
0
votes
0
answers
31
views
How to sample from a given accessible PDF?
I am learning Gibbs Sampling for GMMs. Particularly, given $\boldsymbol \theta$, I must sample from the latent $\boldsymbol z$ before sampling $\boldsymbol x$.
The PDF of $\boldsymbol z$ is given as$$
...
- 601
0
votes
0
answers
21
views
Derive Gibbs sampler for exponential state space model
I have a hidden Markov model.
$X_n = e^{-b\times dt}\times X_{n-1} + \sigma_v \times V_v$
$Y_n = X_n + \sigma_w \times V_w$
where $b$ is the model parameter, $\sigma_w$ is the standard deviation ...
- 9
2
votes
1
answer
72
views
Question on Gibbs sampler, estimate the density function
I am reading the paper Explaining the Gibbs Sampler(Casella, George, and Edward I. George. "Explaining the Gibbs sampler." The American Statistician 46.3 (1992): 167-174). I am stuck with ...
- 4,648
0
votes
0
answers
25
views
How to calculate the likelihood for new component? in Gibbs sampling for DPMM
I'm using Gibbs sampling for the posterior inference of Dirichlet Process Mixture Model (DPMM). During the sampling process for $c$, the category assignment variables, we have
$$P(c_i=c|c_{-i},y_i,\...
- 1
1
vote
1
answer
70
views
Gibbs sampling for one variable in a row
In the gibbs sampling, if I sample only one variable repetitively (say 3 times) while other variables remain the same, and sample next one variable for 3 times and so on..
$x_{1_1}^* \sim p(x_1^* |...
- 45
1
vote
0
answers
55
views
What is "critical slowing down" using MCMC for a Gibbs-Boltzmann distribution?
When sampling from a probability density function of the form $$p(x)=e^{-\beta E(x)},$$ where $E$ is considered to be the energy of a system and $\beta=1/T$ is the inverse of a temperature parameter $...
- 71
2
votes
0
answers
86
views
Why is it easy for the Gibbs sampler to take long time to converge to target distribution?
This is related to Gelman's Bayesian Data Analysis 3rd Edition pg 300 first paragraph of Section 12.4. The book says the following.
"An inherent inefficiency in the Gibbs sampler and Metropolis
...
- 1,365
1
vote
1
answer
87
views
Image denoising with Gibbs sampler
I have a question regarding image denoising.
The setup:
Consider the lattice $L:=\{1,...,m\}^2$ and a process $X=\{x_a\}_{a\in L}$ with $x_a = \pm 1$. Let the observed image be $Y = \{y_a\}_{a\in L}$ ...
0
votes
0
answers
19
views
Expression / R code for theconditional posterior PDF for ordinal regression model (for Metropolis-Hastings step within Gibbs sampler)
I am trying to derive an expression for the PDF (conditional posterior density) of the cutpoint parameters of an ordinal regression model so that I can use a Metropolis-Hastings step to sample the ...
- 11
3
votes
1
answer
93
views
Uniform ergodicity of a Gibbs sampler
We consider a classical data set from Gelfand and Smith containing the information about ten nuclear power plant pump failures. We are interested in the failure intensity of each pump and we employ ...
1
vote
0
answers
134
views
Ordinal regression - 'induced Dirichlet' conditional posterior distribution
I am trying to implement the 'induced Dirichlet' prior model proposed by Michael Betancourt (from section 2.2 of his ordinal regression case study here: https://betanalpha.github.io/assets/...
- 11
0
votes
0
answers
42
views
How to use Gibbs sampler to simulate normal-normal hierarchical models?
This is related to Gelman's BDA 3rd Edition Chapter 11, Sec 3. The book says the following.
"The Gibbs sampler is the simplest of the Markov chain simulation algorithms, and it is our first ...
- 1,365
0
votes
0
answers
39
views
Using Gibbs sampling to verify the analytical solution of 2D Ising Model
Suppose a 2-D Ising model on a period lattice $L\times L.$ I want to apply Gibbs sampling to verify the analytic solution of spontaneous magnetization in 2D Ising model given the Hamiltonian $H=-J\...
- 1
0
votes
1
answer
63
views
Convolution of two multivariate guassian distribution for the posterior predictive distribution
To find the full conditional distribution of $\eta$ for a Gibbs sampling algorithm , I have to show that
$$
p(\eta|-) \propto \int N(\eta;\Omega(\Lambda+\Delta^{-1}\mu),\Omega) N(\mu;\hat{\mu},\Delta/\...
- 1
2
votes
0
answers
70
views
The guidelines for choosing different MCMC algorithms [closed]
MCMC has several types of algorithms: Metropolis-Hastings, Gibbs, Adaptive MH, Hamiltonian Monte Carlo. What are their respective pro/cons, and how to choose them in the Bayesian analysis?
- 5,084
0
votes
0
answers
58
views
Adaptive metropolis within Gibbs
I once saw some study claims to use the algorithm referred to as "Adaptive Metropolis within Gibbs". Are there any formal introduction on this algorithm? What are the difference between this ...
- 3,007
1
vote
1
answer
97
views
Changing scan order of Gibbs Sampler on each iteration
I'm implementing an algorithm that requires the use of Gibbs Sampling and, due to the nature of the way I store the values, it would be efficient to change the order of the updates on each component ...
- 31
0
votes
0
answers
18
views
How does blocked Gibbs Sampling change the interpretation of the generative author-topic (LDA) model
The author topic model is a version of a Latent Dirichlet Allocation model which looks to estimate a set of author to topic, and topic to word distributions to model how authors combine to produce ...
- 11
2
votes
1
answer
79
views
Slice sampling in Particle Gibbs with Ancestral Sampling
Bear with me as I am not from statistical background. My question is about the implementation of PGAS algorithm as given in Lindsten et. al 2014 concerning sampling in state-space models. The two ...
- 121
1
vote
1
answer
129
views
How to use Gibb's sampling when the conditional probability doesn't depend on the observations [closed]
I have a model that looks like this
$$ x(k) = \sum_{m}^{M} e^{i (U_m k + \beta_m)} + n(k)$$
Where $U_m$ has a Gaussian distribution with parameters $\mu$ and $\sigma^2$.
$$ U_m \sim \mathcal{N}(\mu, \...
- 113
0
votes
0
answers
34
views
How much randomness is required for Gibbs sampling?
I am attempting to parallelize a program that executes hundreds of calls to Mallet's getSampledDistribution method, which is essentially an execution of Gibbs sampling over a topic distribution which ...
1
vote
1
answer
87
views
Is it possible to estimate the parameters of a superposition of Poisson processes through Bayesian inference from a binarized sequence?
My question is complementary to a previous problem : Bayesian inference on binarized Poisson distribution. I retake the previous notations. Problem description :
I am counting the number of balls ...
- 11
2
votes
1
answer
124
views
Can the proposal distribution for Metropolis-Hastings within Gibbs be conditioned on other variabless?
I am drawing samples from my posterior, $P(x,y|z)$, using Gibbs sampling. When I sample $x$, I use a Metropolis-Hastings step. My question is whether I am allowed to use a proposal distribution for $x'...
- 123
2
votes
1
answer
248
views
Distribution of conditional posterior for Gibbs sampling
The following is a description of how the authors (Yongning Wang & Ruey S. Tsay) of this (2019) paper Clustering Multiple Time Series with Structural Breaks want to perform Gibbs sampling to ...
1
vote
0
answers
109
views
Mean of skew normal distribution with normal prior obtained with Gibbs sampling
I would like to obtain a new mean $\mu$ of a skew normal distribution with a normal prior of the form $N(\delta,\tau)$ on $\mu$, and a given standard deviation $\sigma$ and shape parameter $\alpha$. ...
- 11
2
votes
2
answers
273
views
Bayesian updates for Dirichlet-multinomial with Gamma prior
Let
$$
\begin{aligned}
X_i &\sim \text{Dir-multinom}(X\mid\lambda)\\
\lambda_{j} &\sim \text{Gamma}(\lambda_j\mid\alpha,\beta)\\
\end{aligned}
$$
where $i$ iterates over observations, $j$ ...
- 245
2
votes
0
answers
45
views
Recovering samples from a density estimation with an additional prior on the samples. Used for Gibbs sampling
Abstract Idea:
Given a noisy measured density ($d_j$ at position $p_j$) and a density model, sample from the model parameters under the following stochastic model:
Stochastic Model:
Prior for model ...
- 21
0
votes
0
answers
169
views
Hyperprior in Gibbs Sampling
Following up from this question, I have managed to derive the following posterior distributions
$$ \lambda_z | \boldsymbol{y}, \Theta^{(-\lambda_z)} \sim Gamma(a + \sum_{i=1}^{n_z} y_{ij}, \quad a + ...
- 1
0
votes
1
answer
520
views
Poisson-Gamma Hierarchical Model
I am fairly new to Gibbs Sampling and I am trying to build a Gibbs Sampler for a Poisson-Gamma hierarchical model.
In this model, there are $m$ restaurants in a city, with $n_z$ number of observations ...
- 1
0
votes
1
answer
121
views
Question about paper on Bayesian Shrinkage Estimation
I am reading the paper Bayesian Shrinkage Estimation of the Relative Abundance of mRNA Transcripts Using SAGE, and I am trying to work out the calculations for the complete conditionals for the Gibbs ...
user310180
2
votes
0
answers
538
views
Gibbs Sampling - why converge to stationary distribution
Currently, I am going through Chapter 12.3 of Probabilistic Graphical Models - Principles and Techniques which talks about MCMC sampling methods. In Chapter 12.3.4.1, it states the following theorem:
...
- 21
1
vote
2
answers
380
views
Question about a mixture dirichlet MCMC model
I am self-learning Bayesian statistics using the book Computational Bayesian Statistics by Turkman et al. and I am currently stuck on Chapter 6 Problem 10. It can be found here on page 124.
I am ...
user310180
0
votes
1
answer
542
views
Beta-Binomial Gibbs Sampler
I am self-studying Bayesian statistics from the book Computational Bayesian Statistics by Turkman et al, but I am stuck on Problem 6.3 from the book:
Suppose we want to consider a Binomial (unknown $\...
user310180
0
votes
1
answer
319
views
Metropolis-Hastings algorithm for logarithmic probability density
Similar question to posted here: Metropolis-Hastings using log of the density however my question is around sampling a random number from a uniform distribution.
I am following the steps outlined in ...
- 5
0
votes
1
answer
386
views
Why Gibbs Sampling for mixture models?
I am studying MCMC and in the book I'm reading there is this example on Gibbs algorithm for inferring the posterior of a gaussian mixture. I understand how the algorithm works and the fact that its ...
0
votes
0
answers
195
views
Bayesian analysis example with convergence under Gibbs but not Metropolis-Hastings
Having a conceptual understanding of algorithms such as Metropolis-Hastings, Gibbs and Hamiltonian Monte Carlo can provide ideas of remediation to apply when models do not converge. This question ...
- 534
1
vote
0
answers
16
views
Is it possible to use a Gibbs sampler for uncertainty propagation?
Situation:
10 basic numeric properties are predicted using quantile regression forest, then they are put into a desicion rule system to decide land management. The desicion rules result in one class (...
- 11
0
votes
0
answers
131
views
Gibbs sampling from a 2D Gaussian
Suppose $\textbf x \sim\mathcal N\left(\begin{pmatrix}1\\1\end{pmatrix}, \begin{pmatrix}1&-1/2\\-1/2&1\end{pmatrix}\right)$. Derive the full conditionals $p(x_1|x_2)$ and $p(x_2|x_1)$. ...
user308286