The Gibbs sampler is a simple form of Markov Chain Monte Carlo simulation, widely used in Bayesian statistics, based on sampling from full conditional distributions for each variable.

learn more… | top users | synonyms

0
votes
0answers
12 views

Gibbs Sampling Parallel convergence

I have a Gibbs Sampler. I am running 1000 parallel walks of the sampler. For any given walk I use the 2 value Kolmogorov Smirnov test to determine if convergence is reached within that instance. I ...
1
vote
0answers
16 views

Why is the posterior the stationary distribution of a Gibbs chain?

I'm having trouble understanding the setup here. I'm follow Probabilistic Graphical Models by Koller and Friedman. They say that we wish to generate samples from the posterior distribution ...
1
vote
0answers
26 views

Efficiency in Metropolis Vs Gibbs sampling

I have read that Gibbs sampling is more efficient than Metropolis algorithm. Why? Is this due only to the fact the in Gibbs sampling the acceptance rate is $1$, so that the chain needs fewer ...
1
vote
0answers
20 views

What does it means of Normalization term of Gibbs distribution?

I am studying about Gibbs distribution concept and I am confusing a one term in that concept that is normalization term. According to the Hammersley–Clifford theorem, an random $x$ can equivalently be ...
0
votes
0answers
26 views

Using BUGS/JAGS for Gibbs sampling for inference in a (discrete) Bayesian Network for estimating conditional probability tables

My goal is to use BUGS (more accurately, JAGS) to perform Gibbs sampling as a process for parameter estimation in Bayesian networks that only have discrete random variables. I am using the following ...
1
vote
1answer
21 views

Gibbs sampling for inferring the parameters of a GMM

I came across the following in Kevin Murphy's "a probabilistic perspective on machine learning". I am struggling to understand the derivation of the conditional probability for $z_i$. I tried ...
2
votes
0answers
40 views

Can I use Adaptive MCMC in any setting?

In time series econometrics and finance, most Bayesian authors approximate their models with a Gibbs Sampler, this is especial true for state space models, SV and so forth. The dimensionality of ...
0
votes
0answers
11 views

How does Shuffled-Complex-Evolution-Metropolis algorithm compare to other adaptive samplers (e.g. NUTS)?

I recently heard of the Shuffled-Complex-Evolution-Metropolis algorithm and am curious how it compares to other adaptive MCMC sampling algorithms. Unfortunately I am still learning about optimizing ...
0
votes
0answers
17 views

Detecting Gibbs Sampler convergence with Raftery and Lewis Diagnostic

Hi! I'm trying to understand and implement the Raftery and Lewis Diagnostic for detecting the number of iterations required for a gibbs sampler but cant seem to understand the formula. ...
0
votes
1answer
33 views

Could anyone help me check my gibbs sampling code? [closed]

I am now trying to write a Gibbs Sampling code based on the posteriors from a paper "Bayesian Regularization via Graph Laplacian", writer: Fei Liu, et. When I run the code, it always show the error: ...
2
votes
0answers
32 views

Gibbs sampling version for estimating Hierarchical Double Dirichlet Process Mixture of Gaussian Processes

I'm trying to implement Gibbs sampling to estimate the parameters of the following non-parametric model: $$\begin{align*} \beta|\gamma & \sim \text{GEM}(\gamma)\\ k_t|\beta & \sim \beta\\ ...
1
vote
0answers
22 views

How do I check or validate the RBM (Restricted Boltzmann Machine) Model?

I'm trying to implement RBM, then i used play tennis case to test the rbm. I've tried autoencoder before, and the result was good. Actually, I confuse with the function of RBM it self, i think it ...
2
votes
2answers
234 views

Canonical example to understand Gibbs Sampling

I'm been trying to understand Gibbs sampling. What I'm looking for is a paper or other reference which uses a simple canonical example and uses that to illustrate Gibbs sampling. Sadly I've not ...
5
votes
1answer
189 views

Bayesian regression full conditional distribution

I have a problem with the derivation of the full conditional distribution of the regression coefficients in a simple Bayesian regression. The source of the following equations is: Lynch (2007). ...
0
votes
0answers
29 views

Conditional density of topic assignment in A Split-Merge MCMC Algorithm for the Hierarchical Dirichlet Process

I'm trying to implement the algorithm described in A Split-Merge MCMC Algorithm for the Hierarchical Dirichlet Process by Chong Wang and David Blei. Equation (7) on page 4 has the terms ...
0
votes
0answers
19 views

Diagnostics for Gibbs sampler in R package topicmodels

I'm experimenting with topic models for my master thesis. I have a dataset consisting of about 300 variables representing words and 900 cases. It's not in document form, because I pre-selected certain ...
3
votes
0answers
221 views

How to understand Gibbs distribution

I have a graph model such as Following the Hammersley–Clifford theorem describes that Markov random fields exhibit a Gibbs distribution with an energy function as follows: $$P(x)=\frac ...
1
vote
2answers
35 views

How to find conditional distributions from joint

I want to learn about how to do Gibbs sampling, starting with finding conditional distributions given a joint distribution. While looking for examples, I found this blog post that I wanted to ...
0
votes
0answers
35 views

Can one use the RTS (Rauch–Tung–Striebel) smoother to simulate latent factors (as opposed to Carter and Kohn procedure)?

It is a little surprising that I have not found anything online in the literature which clarified this. I am working on an MCMC Gibbs sampling procedure to calibrate a "dynamic factor model". One of ...
2
votes
1answer
41 views

Gibbs sampler for a particular distribution

I'm trying to implement Gibbs Sampler for the distribution: $$\pi(x,y)=e^{-10(x^2-y)^2-(y-1/4)^4}$$ So, like the first step, I need to find: $$\phi(t) = \int_{-\infty}^{t} e^{-10(x^2-y)^2-(y-0.25)^4} ...
1
vote
1answer
72 views

In Bayesian analysis, how to sample from full conditional given uniform prior and normal data likelihood?

[EDIT] This question comes from the example of OpenBUGS manual: Stagnant: a changepoint problem and an illustration of how NOT to do MCMC! I also asked another question regarding this example. ...
1
vote
0answers
42 views

In MCMC simulation, how to deal with very small likelihood values that couldn't be represented by computer? [duplicate]

I am working on a Bayesian project based on Stagnant data from a OpenBugs example, which is a changepoint problem. Basically we assume a model with two straight lines that meet at a certain ...
2
votes
0answers
33 views

full conditional posteriors for bayesian lasso

I am reading the original Bayesian Lasso paper, and its follow up; They look straightforward to implement, mainly because of the conditional posterior probability for the gibbs sampler; however, I ...
0
votes
0answers
46 views

Posterior Conditional on Beta in Bayesian Linear Regression with Factor Analysis

This should be an easy question if you're familiar with the terms involved. I am performing some research using a hierarchical Bayesian regression model that incorporates factor analysis into the Beta ...
2
votes
1answer
37 views

Bayesian Mixture Model Gibbs Sampler for two linear relationships

I am attempting to use a Gibbs Sampler to model a mixture of two groups, where the group membership is defined by a linear relationship conditional on x. Both groups have the same slope and intercept, ...
2
votes
0answers
33 views

Conjugate prior for multivariate with known mean and covariance known to a constant

I have a linear trend model (evolving mean and slope) embedded in a larger state space time series model that I would like to constrain to be a spline. With that assumption, the mean and trend ...
2
votes
1answer
69 views

Gibbs sampling for reducible chain

I am new to Gibbs sampling and I ran into a problem with irreducibility. For the Gibbs sampler to work the Markov chain has to be irreducible. But that assumption is not satisfied in my probability ...
1
vote
1answer
60 views

How to incorporate parameter constraints in Metropolis Hastings

I am working on parameter estimation of GARCH model with Metropolis Hastings. But the results I have got doesn't look reasonable, actually it is quite different from what I have got from Gibbs ...
3
votes
1answer
57 views

Gibbs Sampler - Sample mean convergence

To simulate from the posterior distribution $p(\theta|Y)$ where $\theta = (\mu,\lambda_1,\lambda_2)$, I run a Gibbs sampler to draw approximately random values from $p(\theta|Y)$. This Gibbs sampler ...
5
votes
1answer
41 views

Gibbs Sampler output: how many Markov chains?

When running a Gibbs sampler (for $n=200$ Iterations) with two full conditionals, I get the output $\mathbf{x} = (x_1^{(n)},x_2^{(n)})_{n =1,...,200}$. So $\mathbf{x}$ is the realizations of a Gibbs ...
0
votes
0answers
29 views

generating a sequence of indicators based on variable boundaries in JAGS

Suppose I have a vector of indices 1:N, and data y[1], ..., y[N]. I have three variable center points in ...
2
votes
1answer
51 views

Full conditionals - Gibbs Sampler

i want to draw samples from a 5-dimensional posterior distribution $f(k,\theta,\lambda,b_1,b_2|Y=y)$. From Bayes-Theorem there is the following relationship between posterior and likelihood: ...
2
votes
1answer
71 views

Implementing Gibbs sampler in R from posterior distribution

I am referencing a follow-up idea from something I posted earlier (Zero-inflated Poisson and Gibbs sampling, proofs and sampling). I want to implement the Gibbs sampler, by generating a large ...
3
votes
1answer
125 views

Zero-inflated Poisson and Gibbs sampling, proofs and sampling

I am trying to figure out zip-inflated Poisson (ZIP) model. In this model, random data $X_1, .., X_n$ are of the form $X_i=R_iY_i$, where the $Y_i$'s have Poisson distribution ($\lambda$) and the ...
5
votes
0answers
87 views

Bayesian estimates for Deming regression coinciding with least-squares estimates

Consider the following Deming model with independent replicates : $$x_{i,j} \mid \theta_{i} \sim {\cal N}(\theta_{i}, \gamma_X^2), \quad y_{i,j} \mid \theta_{i} \sim {\cal N}(\alpha+\beta\theta_{i}, ...
0
votes
1answer
88 views

Gibbs Sampler Running Wild

So, I'm setting up a Gibbs Sampler using a multivariate normal model with a Jeffreys prior (working through the Hoff book on my own). There's also missing data to be imputed. I've checked my posterior ...
1
vote
1answer
58 views

Gibbs sampling for concentration parameter in Dirichlet Process Mixture models

Let's assume we have a DP mixture model: \begin{align} G &\sim {\rm DP}({\alpha, H})\\ \theta_i &\sim G \\ x_i &\sim F(\theta_i) \end{align} There are many methods to find the posterior ...
3
votes
2answers
176 views

MCMC chain getting stuck

I am trying to use a Metropolis-within-Gibbs type algorithm to sample $\theta$ and $x$ from the following model. Starting with Bayes theorem I can write: $$ P(\theta, x | y) = \frac{P(y | x, \theta) ...
2
votes
1answer
65 views

Derive the Gibbs sampler for this bivariate distribution

I understand the theory of Gibbs sampling. It is an iterative sampling algorithm that defines, sequence of random variables with the property of a Markov chain. Specifically, I choose any starting ...
0
votes
0answers
81 views

How to include a half-Cauchy prior in a Gibbs sampler for a hierarchical model (Rubin's “8 schools”)?

In the third edition of the popular Bayesian Data Analysis, Gelman et al. discuss a variation on Rubin's "8 schools" problem in which only three schools are considered (p. 131). The authors suggest ...
0
votes
1answer
58 views

PyMC consistently under estimating results found in paper. Possibly not sampling enough?

I have been trying to build confidence in (my ability to correctly use) PyMC by working examples. Namely, I have been working on Chickering and Pearl 1997, and more specifically on their 'artificial' ...
3
votes
1answer
121 views

Confusion in Gibbs sampling

I am self-studying Gibbs sampling from a book. The book introduces metropolis hastings algortihm to generate representative values from a posterior distribution. So we know $p(D | \theta) p(\theta)$ ...
4
votes
1answer
44 views

Why does Slice sampler use the log of the density?

The Slice sampler1 takes as its argument the log of the density to be sampled from. Why is it doing this? A commenter on this question pointed out that it makes no sense to "sample" from the log of ...
0
votes
1answer
111 views

Metropolis-Hastings using log of the density

Does Metropolis-Hastings work with the log of the proposal and the density to be sampled from? That is, say we want to sample from a density $\pi(x)$, using a proposal $q(x|x^{old})$, will the ...
0
votes
0answers
19 views

General state space

is there a clear definition for a "general state space" in the sense of Markovchains ? Is for example $\mathbb{N}$ a general state space because it is countable infinity?
2
votes
2answers
216 views

Clustering methods for unknown number of clusters

Matrix $X=[x_1,...,x_i,...,x_N]$ is a data-set containing $N$ data-points that each data-point $x_i$ is a vector of $D$ dimensions. Each dimension is a feature. The number of clusters ($K$) is ...
0
votes
0answers
16 views

Suggestions of a statistical model with IID data and latent variables

While this might be an unusual request, I am looking for a statistical model with certain properties to test my numerical method on and thought I might ask here. The model ought to have the following ...
4
votes
1answer
116 views

Gibbs Sampler transition kernel

Let $\pi$ be the target distribution on $(\mathbb{R}^d,\mathcal{B}(\mathbb{R^d}))$ which is absolutely continuously wrt to the $d$-dimensional Lebesgue measure, i.e : $\pi$ admits a density ...
5
votes
1answer
191 views

Gibbs Sampler contradiction proof

I want to prove that the systematic scan Gibbs sampler yields an aperiodic chain $X$ on a general state space. Let $\pi$ be the stationary distribution for the resulting chain. Suppose to get a ...
1
vote
0answers
19 views

Efficient generation of graph structured correlated random variables via MCMC/Gibbs

Sometime back I had asked this question about generating correlated random draws based on the correlation structure given by a graph. Link Here The solution there requires to create $n\times n$ ...