Questions tagged [gibbs]

The Gibbs sampler is a simple form of Markov Chain Monte Carlo simulation, widely used in Bayesian statistics, based on sampling from full conditional distributions for each variable or group of variables. The name comes from the method being first used on Gibbs random fields modeling of images by Geman and Geman (1984).

Filter by
Sorted by
Tagged with
1 vote
0 answers
14 views

Mean of skew normal distribution with normal prior obtained with Gibbs sampling

I would like to obtain a new mean $\mu$ of a skew normal distribution with a normal prior of the form $N(\delta,\tau)$ on $\mu$, and a given standard deviation $\sigma$ and shape parameter $\alpha$. ...
user avatar
  • 11
0 votes
0 answers
10 views

Posterior predictive distribution of Gibbs Sampling compared to original data

I have implemented a Gibbs sampler to simulate from the joint posterior: $ln(y_1),...,ln(y_n)|\mu,\sigma^2 ∼_{iid} N(\mu,\sigma^2)$ Where both $\mu$ and $\sigma^2$ was unknown. I have simulated values ...
user avatar
  • 81
1 vote
2 answers
95 views

Bayesian updates for Dirichlet-multinomial with Gamma prior

Let $$ \begin{aligned} X_i &\sim \text{Dir-multinom}(X\mid\lambda)\\ \lambda_{j} &\sim \text{Gamma}(\lambda_j\mid\alpha,\beta)\\ \end{aligned} $$ where $i$ iterates over observations, $j$ ...
user avatar
  • 173
2 votes
0 answers
40 views

Recovering samples from a density estimation with an additional prior on the samples. Used for Gibbs sampling

Abstract Idea: Given a noisy measured density ($d_j$ at position $p_j$) and a density model, sample from the model parameters under the following stochastic model: Stochastic Model: Prior for model ...
user avatar
  • 21
0 votes
0 answers
43 views

Hyperprior in Gibbs Sampling

Following up from this question, I have managed to derive the following posterior distributions $$ \lambda_z | \boldsymbol{y}, \Theta^{(-\lambda_z)} \sim Gamma(a + \sum_{i=1}^{n_z} y_{ij}, \quad a + ...
user avatar
  • 1
0 votes
1 answer
83 views

Poisson-Gamma Hierarchical Model

I am fairly new to Gibbs Sampling and I am trying to build a Gibbs Sampler for a Poisson-Gamma hierarchical model. In this model, there are $m$ restaurants in a city, with $n_z$ number of observations ...
user avatar
  • 1
0 votes
0 answers
16 views

Monte Carlo for Dirichlet Multinomial Model

Problem I am trying to implement Markov Chain Monte Carlo for the Dirichlet Multinomial mixture, described in this reference (where one used the expectation maximization algorithm). The model is as ...
user avatar
  • 1,580
0 votes
1 answer
114 views

Question about paper on Bayesian Shrinkage Estimation

I am reading the paper Bayesian Shrinkage Estimation of the Relative Abundance of mRNA Transcripts Using SAGE, and I am trying to work out the calculations for the complete conditionals for the Gibbs ...
user avatar
0 votes
0 answers
58 views

Gibbs Sampling - why converge to stationary distribution

Currently, I am going through Chapter 12.3 of Probabilistic Graphical Models - Principles and Techniques which talks about MCMC sampling methods. In Chapter 12.3.4.1, it states the following theorem: ...
user avatar
1 vote
2 answers
338 views

Question about a mixture dirichlet MCMC model

I am self-learning Bayesian statistics using the book Computational Bayesian Statistics by Turkman et al. and I am currently stuck on Chapter 6 Problem 10. It can be found here on page 124. I am ...
user avatar
0 votes
0 answers
33 views

How do we show that Gibbs sampling (not just each step) converges to the desired distribution?

So it seems straightforward to show that each step of Gibbs sampling is a special-case of Metropolis-Hastings and satisfies detailed balance, but several people have mentioned that a whole pass of ...
user avatar
  • 407
0 votes
1 answer
178 views

Beta-Binomial Gibbs Sampler

I am self-studying Bayesian statistics from the book Computational Bayesian Statistics by Turkman et al, but I am stuck on Problem 6.3 from the book: Suppose we want to consider a Binomial (unknown $\...
user avatar
0 votes
1 answer
66 views

Metropolis-Hastings algorithm for logarithmic probability density

Similar question to posted here: Metropolis-Hastings using log of the density however my question is around sampling a random number from a uniform distribution. I am following the steps outlined in ...
user avatar
0 votes
1 answer
55 views

Why Gibbs Sampling for mixture models?

I am studying MCMC and in the book I'm reading there is this example on Gibbs algorithm for inferring the posterior of a gaussian mixture. I understand how the algorithm works and the fact that its ...
user avatar
0 votes
0 answers
38 views

Help to Proof of Gibbs sampler acceptance rate

i was wondering if someone could explain how the acceptance rate in the Gibbs Sampling works. In the literature it says that in Gibbs case, the acceptance is always 1 because part of the "...
user avatar
0 votes
0 answers
63 views

Bayesian analysis example with convergence under Gibbs but not Metropolis-Hastings

Having a conceptual understanding of algorithms such as Metropolis-Hastings, Gibbs and Hamiltonian Monte Carlo can provide ideas of remediation to apply when models do not converge. This question ...
user avatar
1 vote
0 answers
9 views

Is it possible to use a Gibbs sampler for uncertainty propagation?

Situation: 10 basic numeric properties are predicted using quantile regression forest, then they are put into a desicion rule system to decide land management. The desicion rules result in one class (...
user avatar
  • 11
0 votes
0 answers
16 views

is there a fmcmc equivalent R package for Gibbs sampling?

I was searching for a package like this fmcmc for Gibbs sampling, but with no luck. I've tried gibbs.met, LearnBayes among others, but I would like to try a few more.
user avatar
0 votes
0 answers
54 views

Gibbs sampling from a 2D Gaussian

Suppose $\textbf x \sim\mathcal N\left(\begin{pmatrix}1\\1\end{pmatrix}, \begin{pmatrix}1&-1/2\\-1/2&1\end{pmatrix}\right)$. Derive the full conditionals $p(x_1|x_2)$ and $p(x_2|x_1)$. ...
user avatar
  • 428
1 vote
0 answers
37 views

Step-size adaptation of NUTS within Gibbs

I am trying to solve a hierarchical problem with a Gibbs sampler. I do not have closed-form expressions for the conditionals, thus I have to use another MCMC method within the Gibbs scheme to sample ...
user avatar
  • 33
0 votes
0 answers
69 views

Does thinning in JAGS/Stan reduce computational time for simulating a chain of a given length?

Question Let's say we have a complicated model whose posterior distribution we want to draw from using MCMC. To do this, we simulate a chain of total length $N=10,000$. For the sake of this question, ...
user avatar
  • 157
0 votes
0 answers
35 views

Derivation of conditional Gamma distribution

I'm reading a paper and hoping to implement the authors particular version of a Gibbs sampler. I can follow their description of the algorithm just fine... but I'm getting lost in the finer details ...
user avatar
  • 348
0 votes
1 answer
112 views

Regarding Gibbs sampling and HMC in fitting Bayesian model, their differences and advantages

I have a question regarding the two MCMC algorithms, Gibbs sampling and Hamiltonian Monte Carlo (HMC) for performing the Bayesian analysis. If using Gibbs sampling, my understanding is that we need to ...
user avatar
  • 4,672
0 votes
0 answers
30 views

Bayesian multivariate regression with common coefficients

In a hierarchical model I'm working on, I have $K$ different $N\times P$ predictor matrices, each denoted $X_k$ and $K$ length $N$ outcome vectors each denoted $y_k$. Essentially, I have a ...
user avatar
0 votes
2 answers
82 views

If the prior and likelihood not be conjugate, how to get conditional distribution to sample from using Gibbs sampling?

I know that when prior is conjugate with the posterior, by writing the loglikelihood and log prior and eliminate the non-independent terms for each parameter one can get the conditional distribution ...
user avatar
  • 133
1 vote
1 answer
106 views

Conditional distribution of the weight of a mixture gaussian with data augmentation using gibbs sampling

This question is relate to Differenciate between two distributions using gibbs sampling . For $t=1,\,\dots,\,n$, let's $r_t\sim\mathcal{N}(0,\,\sigma_t^2)$ and $$\sigma_t^2=\left\{\begin{array}{lcl} \...
user avatar
0 votes
1 answer
54 views

Differenciate between two distributions using gibbs sampling [closed]

This question is relate to the post : " Conditional distribution for Gibbs sampling for Gaussian mixture " but is a little bit different. My objective is to know why the algorithm (which is ...
user avatar
0 votes
0 answers
21 views

Measure to capture within-chain fluctuations in MCMC?

I am using two kinds of updates for a particular parameter in MCMC estimation of my model. First update gives the following trace plot: Second update gives the following trace plot: Note that the ...
user avatar
0 votes
0 answers
29 views

Why do we need unary terms in Ising model (pairwise Markov random field)?

Ising model contains both $\phi(i,j)$ and $\phi(i)$. For example, consider a Markov random field with only two nodes $i$ and $j$, if $P=\phi(i,j) * \phi(i) * \phi(j)$, then we can also write $P=\phi(i,...
user avatar
0 votes
0 answers
18 views

Incorrect inference of conditional density in simple linear model and Gibbs sampling

I am studying Bayesian inference and just found out that I have a major misunderstanding of the core concepts behind theory of probability and linear models. Assume the following model: $$y = \epsilon$...
user avatar
  • 77
1 vote
1 answer
169 views

Applying outlier adjustment using student's t distribution in a state-space model

I'm exploring performing outlier adjustment in a state-space model by using student's $t$ distribution. The gist of the problem is formulated as follows: $$ \begin{align*} y_t^* &= u_t + o_t - o_{...
user avatar
2 votes
1 answer
1k views

Gibbs Sampler for Normal and Inverse Gamma Distribution in R

I'm trying to implement a Gibbs sampler for the following conditional distributions using R: This is the code I have in R so far: ...
user avatar
  • 23
0 votes
0 answers
47 views

Posterior conditionals for bivariate normal

I'm new to bayesian statistics and I'm studying Bayesian models. I'm having trouble writing the Gibbs sampler for a particular case of bivariate normals. Assume now that $d_i = (x_i, y_i)$ for $i=1,2\...
user avatar
1 vote
0 answers
48 views

MCMC algorithm for Hierarchical Bayes model with variable number of mixture components

I am trying to develop an MCMC algorithm for clustering $n$ data-points $y_{1},y_{2},\dots,y_{n}$ using a Gaussian mixture model, but with a prior defined on the number of components K. The ...
user avatar
1 vote
0 answers
16 views

Gibbs sampling step for variables that have a complex offline prior in an MCMC hybrid

I have a question about how to use an offline function as a prior when performing a Gibbs/hybrid analysis. Let's say I have data $y$ and some parameters which I'll simplify to $\theta_1, \theta_2$. ...
user avatar
  • 181
1 vote
1 answer
70 views

Expanding conditional probability for Gibbs sampling with many parameters

I'm trying to use Gibbs sampling to get the following target distribution: $$ p(a,b,c \lvert x, z) $$ Where $z = f(x,a,b,c)$ and the rest are independent. I know the following conditional ...
user avatar
2 votes
1 answer
59 views

Why iterations of Gibbs sampling for a bivariate Gaussian distribution can be seen as random walk?

In Section 4.4 of the excellent technical report Probabilistic Inference using Markov Chain Monte Carlo Methods, the author tries to analyze the performance of Gibbs and Metropolis algorithm with ...
user avatar
  • 5,112
1 vote
0 answers
18 views

Plotting a random walk on R [closed]

I've run a Gibbs sampler and obtained a sample for $X_1$ and $X_2$. I'm trying to recreate a plot like this one: How do I recreate the walk part on R?
user avatar
  • 556
0 votes
1 answer
284 views

How to built Gibbs sampler of Mixture Bayesian regression in R?

I am working on a Gibbs sampler of three parameters and we know the full conditional distribution of three parameters.
user avatar
  • 262
1 vote
0 answers
33 views

Particle Gibbs Sampler For Regime-Switching Nonlinear Gaussian SSM

I'm reading this paper on using a non-linear Gaussian SSM for measuring regime-switching leverage effect using stock market data. I'm using it as jump-off point for an undergraduate paper. My advisor ...
user avatar
2 votes
0 answers
46 views

Gibbs updating algorithm (Gibbs steps) for computationally expensive likelihood

I am looking for a good way to update steps in a Gibbs sampler where the likelihood function is computationally expensive. Here is what I tried so far: By default JAGS uses a slice sampler. However, ...
user avatar
  • 2,349
0 votes
0 answers
44 views

Deriving a full conditional distribution when Half-t distribution is used

I'm trying to use a Half-t distribution with Gibbs sampling to form a model. but having hard time finding the full conditional distribution. Suppose $$Y = X \beta + \epsilon $$, where $Y$ is a ...
user avatar
1 vote
0 answers
55 views

Gibbs sampling Bayesian conditional distribution for mean of a Normal distribution

first post here in CV. I'm currently working on a textbook exercise on Gibbs Sampling and got stuck on naming the distribution for one of the conditional distributions. Question Consider a normal ...
user avatar
2 votes
0 answers
61 views

Gibbs sampler of a generative model

I understand what a Gibbs sampler is and I understand how LDA does classification. But I'm unsure how I can generate a Gibbs sampler for an LDA model and how to meld the two concepts. Let's say I ...
user avatar
  • 1,452
4 votes
1 answer
199 views

Bayesian mixture model joint posterior

I am just starting to learn about bayesian mixture models. There is a few clarifications that I want to make which I am not sure myself. The graphical model below describes a gaussian mixture model ...
user avatar
  • 886
4 votes
1 answer
282 views

Implementation of a blocked Gibbs sampler for a mixture model with a Dirichlet-process prior

I am trying to understand and implement the blocked Gibbs sampler described on page 552 in Bayesian Data Analysis by Gelman et al. in the context of using a Dirichlet process as a prior in a mixture ...
user avatar
  • 468
1 vote
2 answers
489 views

Deriving full conditionals from joint distributions?

In this link (https://www.youtube.com/watch?v=a_08GKWHFWo), the author derives the conditional distributions from the joint; but I got lost in the mechanics of what happened, the process was overly ...
user avatar
  • 107
1 vote
0 answers
238 views

Gibbs sampling proposals for bivariate normal?

I'm very familiar with Metropolis-Hastings, having implemented the algorithm myself to handle "toy problems." Gibbs sampling, however, is a bit trickier for me as I'm not quite certain what ...
user avatar
  • 107
2 votes
0 answers
80 views

Intuition on why Gibbs Sampling samples from the posterior distribution

I am new to Gibbs Sampling and I do understand how the algorithm works but I would also like to understand how sampling from the conditional distributions is equivalent to sampling from the joint. ...
user avatar
  • 886
1 vote
0 answers
34 views

How does pymc3 posterior simulation work in this simple case without having the full conditional distributions?

I'm trying to estimate the posterior distribution of the gamma parameters alpha and beta given that my data comes from a gamma distribution and the priors I chose come from two uniform distributions. ...
user avatar

1
2 3 4 5
8