The Gibbs sampler is a simple form of Markov Chain Monte Carlo simulation, widely used in Bayesian statistics, based on sampling from full conditional distributions for each variable.

learn more… | top users | synonyms

4
votes
1answer
44 views

slice sampling within a Gibbs sampler

Questions My questions are: Is the following slice-sampling-within-Gibbs approach valid? Is there a good reference out there that uses, or better yet, justifies it? Context I'm trying to sample ...
2
votes
2answers
58 views

Gibbs sampler gets stuck in local mode

I am very new to statistics and trying to implement a Gibbs sampler. However, according to wikipedia https://en.wikipedia.org/wiki/Gibbs_sampling and this discussion thread ...
0
votes
0answers
24 views

Why sampling for inference of a new document in LDA?

Given a standard LDA model with few 1000 topics and few millions of documents, trained with Mallet / collapsed Gibbs sampler: When inferring a new document: Why not just skip sampling and simply use ...
1
vote
0answers
30 views

How to sample random variables (x,y) from a bivariate Cauchy distribution using a Gibbs sampler?

A bivariate Cauchy distribution is equivalent to a bivariate t-distribution with 1 degree of freedom.
7
votes
0answers
100 views

How to test if a cross-covariance matrix is non-zero

The background of my study: In a Gibbs sampling where we sample $X$ (the variable of interests) and $Y$ from $P(X|Y)$ and $P(Y|X)$ respectively, where $X$ and $Y$ are $k$-dimensional random vectors. ...
1
vote
1answer
22 views

Deriving Gibbs sampler for specific mixture model

Let $\theta_i$ be an indicator which is $0$ if score, $X_i$, is the same for both opponents, $1$ if different: $X_i|\theta_i \stackrel{\text{ind}}{\sim} (1-\theta_i) U(0, 1) + \theta_i Beta(1, ...
5
votes
1answer
40 views

Why does sampling from the posterior predictive distribution $p(x_{new} \mid x_1, \ldots x_n)$ work without having to average out the integral?

In a Bayesian model, the posterior predictive distribution is usually written as: $$ p(x_{new} \mid x_1, \ldots x_n) = \int_{-\infty}^{\infty} p(x_{new}\mid \mu) \ p(\mu \mid x_1, \ldots x_n)d\mu $$ ...
1
vote
0answers
14 views

Estimation of arithmetic Brownian motion volatility with transformed data

I want to estimate the volatility $\sigma$ of a process $(X_t)$ following an arithmetic Brownian motion, that is, for a constant time step $\Delta$, $X_{t+\Delta} = X_t + \sigma B_{\Delta}$ , where ...
1
vote
1answer
67 views

Gibbs sampling from a complex full conditional

I have a sampling question relating to Gibbs sampling of a complicated full conditional. Supposed I have a complicated full conditional that I want a single sample from $p(\theta_i$|$\theta_{-i}$, ...
1
vote
0answers
33 views

How does one determine which variables can be collapsed in Gibbs Sampling?

I am going through the derivation for Gibbs Sampling update equations for LDA. The claim is that $\theta_{d}$ (document specific topic distribution) and $\phi_k$ the topic-word distribution can be ...
0
votes
0answers
15 views

Interpretation of Gibbs sampling in Dirichlet Process posterior calculation

I wonder if anyone who is familiar with Gibbs sampling in the context of Dirichlet Process semi-parametric models could please help clear this question up. In Radford M. Neal's 2000 paper "Markov ...
2
votes
1answer
42 views

Efficiently sampling from mixture distribution posterior

I have the following model: $$ \begin{align} \pi_1\sim & \text{Unif}(0,1)\\ \lambda_1,\lambda_2\sim & \text{Ga}(1,1)\\ z_i\sim & \pi_1^{1(z_i=1)}\pi_2^{1(z_i=2)}\\ ...
1
vote
1answer
12 views

Is it possible for iterations to spike in Gibbs sampling?

After performing Gibbs sampling, I looked at a trace plot for one of my parameters and it appeared to spike at certain points. Is this possible or is it likely that I just coded my sampler wrong?
0
votes
1answer
22 views

Truncated prior leads to non-intuitive posterior

I am setting up a linear regression model for continuous data that is Normally distributed. For this model, I want to assume that my $\beta$ predictor is truncated to be positive, that is $$\beta \sim ...
0
votes
0answers
25 views

Is it possible for Metropolis sampling to converge to the wrong value?

I have simulated data under three parameters of interest, say a, b, c. The prior I put on c was a Gamma, so it only takes positive values. The full conditionals of a and b are known distributions, but ...
0
votes
1answer
32 views

Metropolis sampling (symmetric proposal distribution)

Can Metropolis sampling be used in conjunction with Gibbs sampling? So for example, if I have three parameters of interest, but only two of them have full conditionals that are known distributions, ...
0
votes
0answers
7 views

Gibbs within Collapsed Gibbs?

I have a model with variables $X_{1}, X_{2}, X_{3}, X_{4}$. I would like to sample it within a larger MCMC chain using: $(X_{1}, X_{2}) \sim P(X_{1}, X_{2})$ $(X_{3}, X_{4}) \sim P(X_{3}, X_{4} \mid ...
2
votes
2answers
175 views

Markov Chain Monte Carlo (MCMC) with transformed data

I want to obtain an estimate of a parameter $\Theta$ in a model for a random variable $X$ dependent on $\Theta$ with known but complicated likelihood $L(\Theta|X) = p(X|\Theta)$. $X$ is not directly ...
0
votes
0answers
23 views

Gibbs sampler for the parameters of a truncated normal distribution

I try to set up a Gibbs sampler in a Bayesian approach for the parameters of a truncated normal. But I can't find the full conditional distributions because of the normalization ...
0
votes
0answers
22 views

plot log likelihood function evolution in mcmc simulations

Is it possible to plot log likelihood function evolution in mcmc simulations? I have a mixture model and its parameters are estimated using the gibbs sampling method in r environment and using the ...
2
votes
1answer
99 views

Gibbs sampling with mixed prior using a Metropolis-Hastings step

My questions are about a sampling procedure for fitting a Bayesian hierarchical model where one of the priors is a mixture distribution of discrete and continuous parts. The model is not my own but I ...
1
vote
2answers
84 views

Gibbs sampler implementation

I am just getting started with the Gibbs Sampler and came across an implementation from here and here and here. All of theses implementations are based on the first article. There is an inner loop ...
0
votes
1answer
23 views

Gibbs sampling: ancillary and sufficient parametrization

After asking a question about Gibbs sampling earlier, I have another one for you. I have not been able to find laymen's background on this, the only referenced use I've found for this is in ...
3
votes
1answer
88 views

Gibbs sampling convergence

In an astronomical context, the authors of a paper desire to use a Gibbs algorithm. Please note: I am inexperience in MCMC algorithms, and specifically in Gibbs sampling. What we want, in essence, is ...
0
votes
0answers
23 views

Gibbs Sampling Parallel convergence

I have a Gibbs Sampler. I am running 1000 parallel walks of the sampler. For any given walk I use the 2 value Kolmogorov Smirnov test to determine if convergence is reached within that instance. I ...
1
vote
0answers
33 views

Why is the posterior the stationary distribution of a Gibbs chain?

I'm having trouble understanding the setup here. I'm follow Probabilistic Graphical Models by Koller and Friedman. They say that we wish to generate samples from the posterior distribution ...
1
vote
0answers
52 views

Efficiency in Metropolis Vs Gibbs sampling

I have read that Gibbs sampling is more efficient than Metropolis algorithm. Why? Is this due only to the fact the in Gibbs sampling the acceptance rate is $1$, so that the chain needs fewer ...
1
vote
0answers
31 views

What does it means of Normalization term of Gibbs distribution?

I am studying about Gibbs distribution concept and I am confusing a one term in that concept that is normalization term. According to the Hammersley–Clifford theorem, an random $x$ can equivalently be ...
1
vote
1answer
39 views

Gibbs sampling for inferring the parameters of a GMM

I came across the following in Kevin Murphy's "a probabilistic perspective on machine learning". I am struggling to understand the derivation of the conditional probability for $z_i$. I tried ...
2
votes
0answers
55 views

Can I use Adaptive MCMC in any setting?

In time series econometrics and finance, most Bayesian authors approximate their models with a Gibbs Sampler, this is especial true for state space models, SV and so forth. The dimensionality of ...
0
votes
0answers
27 views

How does Shuffled-Complex-Evolution-Metropolis algorithm compare to other adaptive samplers (e.g. NUTS)?

I recently heard of the Shuffled-Complex-Evolution-Metropolis algorithm and am curious how it compares to other adaptive MCMC sampling algorithms. Unfortunately I am still learning about optimizing ...
0
votes
0answers
30 views

Detecting Gibbs Sampler convergence with Raftery and Lewis Diagnostic

Hi! I'm trying to understand and implement the Raftery and Lewis Diagnostic for detecting the number of iterations required for a gibbs sampler but cant seem to understand the formula. ...
0
votes
1answer
59 views

Could anyone help me check my gibbs sampling code? [closed]

I am now trying to write a Gibbs Sampling code based on the posteriors from a paper "Bayesian Regularization via Graph Laplacian", writer: Fei Liu, et. When I run the code, it always show the error: ...
2
votes
0answers
57 views

Gibbs sampling version for estimating Hierarchical Double Dirichlet Process Mixture of Gaussian Processes

I'm trying to implement Gibbs sampling to estimate the parameters of the following non-parametric model: $$\begin{align*} \beta|\gamma & \sim \text{GEM}(\gamma)\\ k_t|\beta & \sim \beta\\ ...
1
vote
0answers
47 views

How do I check or validate the RBM (Restricted Boltzmann Machine) Model?

I'm trying to implement RBM, then i used play tennis case to test the rbm. I've tried autoencoder before, and the result was good. Actually, I confuse with the function of RBM it self, i think it ...
2
votes
2answers
245 views

Canonical example to understand Gibbs Sampling

I'm been trying to understand Gibbs sampling. What I'm looking for is a paper or other reference which uses a simple canonical example and uses that to illustrate Gibbs sampling. Sadly I've not ...
4
votes
1answer
295 views

Bayesian regression full conditional distribution

I have a problem with the derivation of the full conditional distribution of the regression coefficients in a simple Bayesian regression. The source of the following equations is: Lynch (2007). ...
0
votes
0answers
43 views

Conditional density of topic assignment in A Split-Merge MCMC Algorithm for the Hierarchical Dirichlet Process

I'm trying to implement the algorithm described in A Split-Merge MCMC Algorithm for the Hierarchical Dirichlet Process by Chong Wang and David Blei. Equation (7) on page 4 has the terms ...
0
votes
0answers
27 views

Diagnostics for Gibbs sampler in R package topicmodels

I'm experimenting with topic models for my master thesis. I have a dataset consisting of about 300 variables representing words and 900 cases. It's not in document form, because I pre-selected certain ...
3
votes
0answers
257 views

How to understand Gibbs distribution

I have a graph model such as Following the Hammersley–Clifford theorem describes that Markov random fields exhibit a Gibbs distribution with an energy function as follows: $$P(x)=\frac ...
1
vote
2answers
46 views

How to find conditional distributions from joint

I want to learn about how to do Gibbs sampling, starting with finding conditional distributions given a joint distribution. While looking for examples, I found this blog post that I wanted to ...
0
votes
1answer
77 views

Can one use the RTS (Rauch–Tung–Striebel) smoother to simulate latent factors (as opposed to Carter and Kohn procedure)?

It is a little surprising that I have not found anything online in the literature which clarified this. I am working on an MCMC Gibbs sampling procedure to calibrate a "dynamic factor model". One of ...
2
votes
1answer
47 views

Gibbs sampler for a particular distribution

I'm trying to implement Gibbs Sampler for the distribution: $$\pi(x,y)=e^{-10(x^2-y)^2-(y-1/4)^4}$$ So, like the first step, I need to find: $$\phi(t) = \int_{-\infty}^{t} e^{-10(x^2-y)^2-(y-0.25)^4} ...
2
votes
1answer
138 views

In Bayesian analysis, how to sample from full conditional given uniform prior and normal data likelihood?

[EDIT] This question comes from the example of OpenBUGS manual: Stagnant: a changepoint problem and an illustration of how NOT to do MCMC! I also asked another question regarding this example. ...
1
vote
0answers
49 views

In MCMC simulation, how to deal with very small likelihood values that couldn't be represented by computer? [duplicate]

I am working on a Bayesian project based on Stagnant data from a OpenBugs example, which is a changepoint problem. Basically we assume a model with two straight lines that meet at a certain ...
3
votes
0answers
46 views

full conditional posteriors for bayesian lasso

I am reading the original Bayesian Lasso paper, and its follow up; They look straightforward to implement, mainly because of the conditional posterior probability for the gibbs sampler; however, I ...
0
votes
0answers
64 views

Posterior Conditional on Beta in Bayesian Linear Regression with Factor Analysis

This should be an easy question if you're familiar with the terms involved. I am performing some research using a hierarchical Bayesian regression model that incorporates factor analysis into the Beta ...
2
votes
1answer
55 views

Bayesian Mixture Model Gibbs Sampler for two linear relationships

I am attempting to use a Gibbs Sampler to model a mixture of two groups, where the group membership is defined by a linear relationship conditional on x. Both groups have the same slope and intercept, ...
2
votes
0answers
45 views

Conjugate prior for multivariate with known mean and covariance known to a constant

I have a linear trend model (evolving mean and slope) embedded in a larger state space time series model that I would like to constrain to be a spline. With that assumption, the mean and trend ...
2
votes
1answer
82 views

Gibbs sampling for reducible chain

I am new to Gibbs sampling and I ran into a problem with irreducibility. For the Gibbs sampler to work the Markov chain has to be irreducible. But that assumption is not satisfied in my probability ...