Skip to main content

Questions tagged [gibbs]

The Gibbs sampler is a simple form of Markov Chain Monte Carlo simulation, widely used in Bayesian statistics, based on sampling from full conditional distributions for each variable or group of variables. The name comes from the method being first used on Gibbs random fields modeling of images by Geman and Geman (1984).

Filter by
Sorted by
Tagged with
0 votes
0 answers
5 views

Proving whether or not a markov chain is irreducible/recurrent? (Metropolis-within-Gibbs)

We want to generate samples from a standard normal distribution using a variation on slice sampling. To do this, the following Gibbs scheme is proposed to sample uniformly from the set $S = \{(u,x)\...
Bajas's user avatar
  • 66
0 votes
0 answers
27 views

Overcoming posterior correlation for a model with random effects (for a Gibbs sampler)

I am trying to infer parameters for a model of case numbers of different infectious diseases in different locations over time. The model is $$ \log \left(1 + y_{ijt}\right)\sim\mathsf{Normal}\left(\mu ...
Till Hoffmann's user avatar
1 vote
0 answers
35 views

Enumerating feasible solutions to the subset sum problem using Gibbs sampling

Given a set of $m$ strictly positive real numbers $W = \{ w_{1}, \dots, w_{m} \}$, I want to find subsets of $W$ whose sum is less than or equal to a maximum value $N$ using Gibbs sampling. To do this,...
scj's user avatar
  • 101
4 votes
1 answer
205 views

How to sample using Gibbs with a uniform latent variable?

I trying to sample using Gibbs from a proportional distribucion $f_{Z}(z)$: \begin{align*} f_{Z}(z) \propto e^{-z}\left(1-e^{-z}\right)^4, \quad z >0 \end{align*} using the joint $f_{Z,\textbf{U}...
daniel's user avatar
  • 155
1 vote
1 answer
263 views

Posterior Distribution in a Bayesian Multivariate Normal Model

I am currently working on a Bayesian inference problem and would appreciate some help on computing the posterior distribution of a hyperparameter within a specific multivariate normal model. Below, I ...
Dalek's user avatar
  • 117
0 votes
0 answers
72 views

Conflicting results from convergence measures for MCMC

I have a Gibbs sampling algorithm, for which I would like to estimate burn-in time. The model isn't hugely complex, and I run sampling for 1000 iterations. One approach I took was tracking the running ...
Blahblahblacksheep's user avatar
1 vote
0 answers
72 views

Metropolis within Gibbs

I have the following variables: $\beta,\Omega,\tau$. $\tau$ is a hyperparameter for the prior on $\beta$. For a fixed $\tau$, I have the conditionals:$p(\Omega|\beta),p(\beta|\Omega)$. To generate ...
user1848065's user avatar
0 votes
0 answers
30 views

Meaning and importance of 'Gibbs update' in MCMC

I am studying MCMC by "Handbook of Markov Chain Monte Carlo" by Brooks, Gelman This book is nice to explaining many fundamental concepts regarding MCMC. Especially in first chapter, they ...
Kyungmin's user avatar
0 votes
1 answer
43 views

Block sampling hidden state using forward algorithm only

In a hidden Markov model, I can't get my mind around why I can't sample the full hidden state $\vec x$ using only a forward sampling algorithm. Let $\vec y$ be the observed data and $\theta$ the model ...
J. Zeitouni's user avatar
0 votes
0 answers
59 views

Bayesian model (Gibbs sampling) with frequency data for non-standard distribution

I have the following model. \begin{align} X &= D\,(F+S\,(1+G)) \\ D &\sim \mathrm{Exp}\,(\lambda) \\ F &\sim \mathrm{Unif}\,(0,1) \\ S &\sim \mathrm{Poi}\,(\sigma) \\ G &\sim \...
jessexknight's user avatar
1 vote
0 answers
21 views

Gibbs sampler, how to generate samples based on odds?

I am reading a paper that contains a Gibbs sampler for a regression model with parameters $\beta$ and design matrix $X$. One of the steps in the Gibbs sampler requires simulating from a binary random ...
Moncho's user avatar
  • 11
0 votes
0 answers
27 views

Can I add more steps to a Gibbs sampler without hurting the ergodicity of the chain?

I have a Gibbs sampler that updates a system of $n$ variables $(x_1,\ldots,x_n)$ by each of the full conditional distributions. Let's say that I add a $n+1^{th}$ step: I also update $v^T(x_1,\ldots,...
SCS's user avatar
  • 11
1 vote
0 answers
62 views

Gibbs sampler: Conditional distribution with nested latent variable distributions

I have the following model (simplified here for the description). The obsvered variable is $y_i$ which is a linear function of some random variable $\eta_i$ and a random error term $\epsilon_i$: $y_{i}...
Stefan's user avatar
  • 165
4 votes
1 answer
410 views

Monte Carlo Options for Data Augmentation

In the seminal paper by Tanner and Wong (1987) on data augmentation, they describe a method for obtaining the posterior distribution $p(\theta|y)$ by data augmentation. Let $Z$ be a latent variable (...
Winston's user avatar
  • 217
0 votes
0 answers
35 views

How to sample from a given accessible PDF?

I am learning Gibbs Sampling for GMMs. Particularly, given $\boldsymbol \theta$, I must sample from the latent $\boldsymbol z$ before sampling $\boldsymbol x$. The PDF of $\boldsymbol z$ is given as$$ ...
wd violet's user avatar
  • 777
0 votes
0 answers
31 views

Derive Gibbs sampler for exponential state space model

I have a hidden Markov model. $X_n = e^{-b\times dt}\times X_{n-1} + \sigma_v \times V_v$ $Y_n = X_n + \sigma_w \times V_w$ where $b$ is the model parameter, $\sigma_w$ is the standard deviation ...
aglheyxc's user avatar
2 votes
1 answer
100 views

Question on Gibbs sampler, estimate the density function

I am reading the paper Explaining the Gibbs Sampler(Casella, George, and Edward I. George. "Explaining the Gibbs sampler." The American Statistician 46.3 (1992): 167-174). I am stuck with ...
Deep North's user avatar
  • 4,776
1 vote
1 answer
110 views

Gibbs sampling for one variable in a row

In the gibbs sampling, if I sample only one variable repetitively (say 3 times) while other variables remain the same, and sample next one variable for 3 times and so on.. $x_{1_1}^* \sim p(x_1^* |...
zzingae's user avatar
  • 45
1 vote
0 answers
114 views

What is "critical slowing down" using MCMC for a Gibbs-Boltzmann distribution?

When sampling from a probability density function of the form $$p(x)=e^{-\beta E(x)},$$ where $E$ is considered to be the energy of a system and $\beta=1/T$ is the inverse of a temperature parameter $...
0xbadf00d's user avatar
  • 303
2 votes
0 answers
217 views

Why is it easy for the Gibbs sampler to take long time to converge to target distribution?

This is related to Gelman's Bayesian Data Analysis 3rd Edition pg 300 first paragraph of Section 12.4. The book says the following. "An inherent inefficiency in the Gibbs sampler and Metropolis ...
user45765's user avatar
  • 1,445
2 votes
1 answer
168 views

Image denoising with Gibbs sampler

I have a question regarding image denoising. The setup: Consider the lattice $L:=\{1,...,m\}^2$ and a process $X=\{x_a\}_{a\in L}$ with $x_a = \pm 1$. Let the observed image be $Y = \{y_a\}_{a\in L}$ ...
Benjamin Bech's user avatar
3 votes
1 answer
131 views

Uniform ergodicity of a Gibbs sampler

We consider a classical data set from Gelfand and Smith containing the information about ten nuclear power plant pump failures. We are interested in the failure intensity of each pump and we employ ...
Alexandria Cortez's user avatar
1 vote
0 answers
199 views

Ordinal regression - 'induced Dirichlet' conditional posterior distribution

I am trying to implement the 'induced Dirichlet' prior model proposed by Michael Betancourt (from section 2.2 of his ordinal regression case study here: https://betanalpha.github.io/assets/...
E_1996's user avatar
  • 11
0 votes
0 answers
75 views

How to use Gibbs sampler to simulate normal-normal hierarchical models?

This is related to Gelman's BDA 3rd Edition Chapter 11, Sec 3. The book says the following. "The Gibbs sampler is the simplest of the Markov chain simulation algorithms, and it is our first ...
user45765's user avatar
  • 1,445
0 votes
1 answer
86 views

Convolution of two multivariate guassian distribution for the posterior predictive distribution

To find the full conditional distribution of $\eta$ for a Gibbs sampling algorithm , I have to show that $$ p(\eta|-) \propto \int N(\eta;\Omega(\Lambda+\Delta^{-1}\mu),\Omega) N(\mu;\hat{\mu},\Delta/\...
Seth22's user avatar
  • 1
2 votes
0 answers
118 views

The guidelines for choosing different MCMC algorithms [closed]

MCMC has several types of algorithms: Metropolis-Hastings, Gibbs, Adaptive MH, Hamiltonian Monte Carlo. What are their respective pro/cons, and how to choose them in the Bayesian analysis?
user3269's user avatar
  • 5,222
0 votes
0 answers
75 views

Adaptive metropolis within Gibbs

I once saw some study claims to use the algorithm referred to as "Adaptive Metropolis within Gibbs". Are there any formal introduction on this algorithm? What are the difference between this ...
user3125's user avatar
  • 3,049
1 vote
1 answer
163 views

Changing scan order of Gibbs Sampler on each iteration

I'm implementing an algorithm that requires the use of Gibbs Sampling and, due to the nature of the way I store the values, it would be efficient to change the order of the updates on each component ...
Edovt's user avatar
  • 31
2 votes
1 answer
111 views

Slice sampling in Particle Gibbs with Ancestral Sampling

Bear with me as I am not from statistical background. My question is about the implementation of PGAS algorithm as given in Lindsten et. al 2014 concerning sampling in state-space models. The two ...
Zero's user avatar
  • 121
1 vote
1 answer
223 views

How to use Gibb's sampling when the conditional probability doesn't depend on the observations [closed]

I have a model that looks like this $$ x(k) = \sum_{m}^{M} e^{i (U_m k + \beta_m)} + n(k)$$ Where $U_m$ has a Gaussian distribution with parameters $\mu$ and $\sigma^2$. $$ U_m \sim \mathcal{N}(\mu, \...
CfourPiO's user avatar
  • 235
0 votes
0 answers
47 views

How much randomness is required for Gibbs sampling?

I am attempting to parallelize a program that executes hundreds of calls to Mallet's getSampledDistribution method, which is essentially an execution of Gibbs sampling over a topic distribution which ...
Fledgling Pidgeon's user avatar
1 vote
1 answer
103 views

Is it possible to estimate the parameters of a superposition of Poisson processes through Bayesian inference from a binarized sequence?

My question is complementary to a previous problem : Bayesian inference on binarized Poisson distribution. I retake the previous notations. Problem description : I am counting the number of balls ...
user361947's user avatar
2 votes
1 answer
199 views

Can the proposal distribution for Metropolis-Hastings within Gibbs be conditioned on other variabless?

I am drawing samples from my posterior, $P(x,y|z)$, using Gibbs sampling. When I sample $x$, I use a Metropolis-Hastings step. My question is whether I am allowed to use a proposal distribution for $x'...
Shep Bryan's user avatar
2 votes
1 answer
256 views

Distribution of conditional posterior for Gibbs sampling

The following is a description of how the authors (Yongning Wang & Ruey S. Tsay) of this (2019) paper Clustering Multiple Time Series with Structural Breaks want to perform Gibbs sampling to ...
user773674's user avatar
1 vote
0 answers
151 views

Mean of skew normal distribution with normal prior obtained with Gibbs sampling

I would like to obtain a new mean $\mu$ of a skew normal distribution with a normal prior of the form $N(\delta,\tau)$ on $\mu$, and a given standard deviation $\sigma$ and shape parameter $\alpha$. ...
Kilian's user avatar
  • 11
2 votes
2 answers
346 views

Bayesian updates for Dirichlet-multinomial with Gamma prior

Let $$ \begin{aligned} X_i &\sim \text{Dir-multinom}(X\mid\lambda)\\ \lambda_{j} &\sim \text{Gamma}(\lambda_j\mid\alpha,\beta)\\ \end{aligned} $$ where $i$ iterates over observations, $j$ ...
Faydey's user avatar
  • 225
2 votes
0 answers
47 views

Recovering samples from a density estimation with an additional prior on the samples. Used for Gibbs sampling

Abstract Idea: Given a noisy measured density ($d_j$ at position $p_j$) and a density model, sample from the model parameters under the following stochastic model: Stochastic Model: Prior for model ...
jan's user avatar
  • 21
0 votes
0 answers
204 views

Hyperprior in Gibbs Sampling

Following up from this question, I have managed to derive the following posterior distributions $$ \lambda_z | \boldsymbol{y}, \Theta^{(-\lambda_z)} \sim Gamma(a + \sum_{i=1}^{n_z} y_{ij}, \quad a + ...
Naja's user avatar
  • 11
1 vote
1 answer
725 views

Poisson-Gamma Hierarchical Model

I am fairly new to Gibbs Sampling and I am trying to build a Gibbs Sampler for a Poisson-Gamma hierarchical model. In this model, there are $m$ restaurants in a city, with $n_z$ number of observations ...
Naja's user avatar
  • 11
0 votes
1 answer
123 views

Question about paper on Bayesian Shrinkage Estimation

I am reading the paper Bayesian Shrinkage Estimation of the Relative Abundance of mRNA Transcripts Using SAGE, and I am trying to work out the calculations for the complete conditionals for the Gibbs ...
user avatar
3 votes
0 answers
828 views

Gibbs Sampling - why converge to stationary distribution

Currently, I am going through Chapter 12.3 of Probabilistic Graphical Models - Principles and Techniques which talks about MCMC sampling methods. In Chapter 12.3.4.1, it states the following theorem: ...
zsliu98's user avatar
  • 31
1 vote
2 answers
398 views

Question about a mixture dirichlet MCMC model

I am self-learning Bayesian statistics using the book Computational Bayesian Statistics by Turkman et al. and I am currently stuck on Chapter 6 Problem 10. It can be found here on page 124. I am ...
user avatar
0 votes
1 answer
700 views

Beta-Binomial Gibbs Sampler

I am self-studying Bayesian statistics from the book Computational Bayesian Statistics by Turkman et al, but I am stuck on Problem 6.3 from the book: Suppose we want to consider a Binomial (unknown $\...
user avatar
0 votes
1 answer
685 views

Metropolis-Hastings algorithm for logarithmic probability density

Similar question to posted here: Metropolis-Hastings using log of the density however my question is around sampling a random number from a uniform distribution. I am following the steps outlined in ...
spacexyz's user avatar
0 votes
1 answer
678 views

Why Gibbs Sampling for mixture models?

I am studying MCMC and in the book I'm reading there is this example on Gibbs algorithm for inferring the posterior of a gaussian mixture. I understand how the algorithm works and the fact that its ...
Federico Butori's user avatar
0 votes
0 answers
224 views

Bayesian analysis example with convergence under Gibbs but not Metropolis-Hastings

Having a conceptual understanding of algorithms such as Metropolis-Hastings, Gibbs and Hamiltonian Monte Carlo can provide ideas of remediation to apply when models do not converge. This question ...
Single Malt's user avatar
1 vote
0 answers
17 views

Is it possible to use a Gibbs sampler for uncertainty propagation?

Situation: 10 basic numeric properties are predicted using quantile regression forest, then they are put into a desicion rule system to decide land management. The desicion rules result in one class (...
Nocci's user avatar
  • 11
0 votes
0 answers
144 views

Gibbs sampling from a 2D Gaussian

Suppose $\textbf x \sim\mathcal N\left(\begin{pmatrix}1\\1\end{pmatrix}, \begin{pmatrix}1&-1/2\\-1/2&1\end{pmatrix}\right)$. Derive the full conditionals $p(x_1|x_2)$ and $p(x_2|x_1)$. ...
user avatar
2 votes
0 answers
197 views

Step-size adaptation of NUTS within Gibbs

I am trying to solve a hierarchical problem with a Gibbs sampler. I do not have closed-form expressions for the conditionals, thus I have to use another MCMC method within the Gibbs scheme to sample ...
Felipe's user avatar
  • 53
0 votes
0 answers
389 views

Does thinning in JAGS/Stan reduce computational time for simulating a chain of a given length?

Question Let's say we have a complicated model whose posterior distribution we want to draw from using MCMC. To do this, we simulate a chain of total length $N=10,000$. For the sake of this question, ...
bschneidr's user avatar
  • 452

1
2 3 4 5
9