Questions tagged [gibbs]

The Gibbs sampler is a simple form of Markov Chain Monte Carlo simulation, widely used in Bayesian statistics, based on sampling from full conditional distributions for each variable or group of variables. The name comes from the method being first used on Gibbs random fields modeling of images by Geman and Geman (1984).

Filter by
Sorted by
Tagged with
5
votes
2answers
810 views

slice sampling within a Gibbs sampler

Questions My questions are: Is the following slice-sampling-within-Gibbs approach valid? Is there a good reference out there that uses, or better yet, justifies it? Context I'm trying to sample ...
5
votes
2answers
3k views

Clustering methods for unknown number of clusters

Matrix $X=[x_1,...,x_i,...,x_N]$ is a data-set containing $N$ data-points that each data-point $x_i$ is a vector of $D$ dimensions. Each dimension is a feature. The number of clusters ($K$) is unknown....
5
votes
1answer
81 views

How to use Gibbs sampling when target function is known only up to normalising constant?

Assume we want to use Gibbs sampling to get an estimate of the parameter , and that we have the following expression for the conditional posterior of : If I am not mistaken, this means that the ...
5
votes
1answer
930 views

Methods of fitting a dynamic linear model

I'm taking a time series course and am learning about exchangeable time series form of dynamic linear models (DLMs). This is given by: \begin{align*} \mathbf{y}_t' &= \mathbf{F}_t'\boldsymbol{\...
5
votes
1answer
2k views

What are the main differences between classical and Gibbs sampling Latent Dirichlet Allocations?

In these weeks I have been studying the classical Latent Dirichlet Allocation (LDA) algorithm by David Blei and colleagues (2003), and the LDA variant based on Gibbs sampling introduced by Tom ...
5
votes
1answer
97 views

Estimation of Bayesian Models

I'm trying to get into Bayesian model estimation (I'm interested in posterior parameter distributions). I could get away with Metropolis-Hastings and Gibbs Sampling for models with few parameters (<...
5
votes
0answers
614 views

Gibbs sampling deriving complete conditionals with mixture priors

My question is about the derivation of the complete conditionals for Gibbs sampling in a hierarchical model where some of the parameters are mixtures of point-masses and Normal distributions. The ...
5
votes
0answers
242 views

Bayesian estimates for Deming regression coinciding with least-squares estimates

Consider the following Deming model with independent replicates : $$x_{i,j} \mid \theta_{i} \sim {\cal N}(\theta_{i}, \gamma_X^2), \quad y_{i,j} \mid \theta_{i} \sim {\cal N}(\alpha+\beta\theta_{i}, \...
4
votes
2answers
405 views

Posterior computation for Laplace distribution

I am dealing with being Bayesian and looking for a closed form for a posterior for the scale parameter $\tau$ of a Laplace distribution, such that I can derive a full conditional in my Gibbs sampler. ...
4
votes
2answers
2k views

How to find conditional distributions from joint

I want to learn about how to do Gibbs sampling, starting with finding conditional distributions given a joint distribution. While looking for examples, I found this blog post that I wanted to ...
4
votes
2answers
1k views

MCMC chain getting stuck

I am trying to use a Metropolis-within-Gibbs type algorithm to sample $\theta$ and $x$ from the following model. Starting with Bayes theorem I can write: $$ P(\theta, x | y) = \frac{P(y | x, \theta) ...
4
votes
2answers
302 views

Canonical example to understand Gibbs Sampling

I'm been trying to understand Gibbs sampling. What I'm looking for is a paper or other reference which uses a simple canonical example and uses that to illustrate Gibbs sampling. Sadly I've not ...
4
votes
1answer
370 views

Gibbs sampling and conditional distribution

I need to simulate the posterior distribution of intraclass correlation coefficient $\pi(\rho|y)$ where $y$ is the data set and $\rho=\frac{\sigma_a^2}{\sigma_a^2+\sigma_e^2}$ with $\sigma^2_a\sim IG(\...
4
votes
2answers
274 views

The Harris recurrence of a stepping-out slice-sampling-within-Gibbs MCMC

I want to use a multistage version of the MCMC here. That is, I want to use a Gibbs sampler to draw from a general joint distribution $p(x_1, x_2, x_3, \ldots)$ with a Gibbs step for each full ...
4
votes
2answers
2k views

A robust R package to do MCMC and Gibbs sampling

I need to make linear model for which I need to do Gibbs sampling in MCMC simulations. The model needed to be fitted is a linear mixed model. Please suggest me for a robust R package for this task.
4
votes
1answer
195 views

handling metropolis hastings rejection during a Gibbs sweep

Suppose I have a MCMC involving a 2 step Gibbs sampler. The first part uses metropolis hastings to find the next parameter value. If during one sweep, the result for the first part is a rejections, ...
4
votes
2answers
900 views

Is Slice Sampling a special case of Gibbs Sampling?

I read on this thread the following: If you can use both the gibbs sampler and slice sampling to sample from a posterior I would use the Gibbs sampler as the slice sampler seems unnecessary to ...
4
votes
2answers
359 views

Resources about Gibbs sampling in hybrid Bayesian networks

I've been trying to get my hands on a substantial resource for using Gibbs sampling in hybrid Bayesian networks, that is, networks with both continuous and discrete variables. So far I can't say I ...
4
votes
1answer
1k views

Explanation regarding Gibbs Sampling

I am new to MCMC and reading a intro paper regarding Gibbs sampling. However, there are two parts in the paper I cannot understand and get stuck. The first part is equation 2.3 in page 168. It says ...
4
votes
1answer
477 views

Reversibility in MCMC

I am reading Geyer's lecture notes on MCMC. A condensed version of these notes constitutes Chapter 1 of the Handbook of Markov Chain Monte Carlo (ed. Brooks et al., 2011). Geyer notes that the ...
4
votes
1answer
38 views

Find joint maximum of a sampled density

I used a sampling method to fit a model with three parameters to data, by supplying the likelihood function and priors. (I'm using JAGS but I think this applies to any method). I obtain triplets of ...
4
votes
1answer
172 views

Gibbs sampling, what to use?

My question concerns Gibbs sampling. Suppose that I have three unknown quantities, $\mu, \sigma^2$ and $c$. I have given prior information and I have given the likelihood which allows me to compute ...
4
votes
1answer
119 views

Gibbs sampler for a particular distribution

I'm trying to implement Gibbs Sampler for the distribution: $$\pi(x,y)=e^{-10(x^2-y)^2-(y-1/4)^4}$$ So, like the first step, I need to find: $$\phi(t) = \int_{-\infty}^{t} e^{-10(x^2-y)^2-(y-0.25)^4} ...
4
votes
1answer
879 views

JAGS, cannot evaluate upper index of counter

I asked this question at the JAGS sourceforge help forum but didn't get response there. I have the following JAGS model: ...
4
votes
1answer
372 views

What is the state-of-the-art regarding sampling from discrete distribution?

After struggling with auto-Poisson model (a.k.a. Random Markov Network with conditional Poisson distributions) trying to force Gibbs sampler to obtain discrete sample of the network (since I know ...
4
votes
2answers
768 views

Is sequential Bayesian updating an option when using MCMC?

I have an implementation of the Griddy Gibbs sampler, but my observations on which I'm conditioning model parameters are too many in number, thus the likelihood underflows quickly, even with a log ...
4
votes
1answer
789 views

In Bayesian analysis, how to sample from full conditional given uniform prior and normal data likelihood?

[EDIT] This question comes from the example of OpenBUGS manual: Stagnant: a changepoint problem and an illustration of how NOT to do MCMC! I also asked another question regarding this example. [...
4
votes
1answer
91 views

Confusion related to derivation of conditional distribution

I was reading this paper http://stat.duke.edu/phd-program/gcc/resources/ResourcesDocuments/CodeExplanation.pdf related to Gibbs sampling. Suppose we have n iid samples $x_i$ from $N(\mu,\sigma^2)$ ...
4
votes
1answer
72 views

Sampling from multivariate normal conditional on a negative minimum

Let $X\sim \mathcal{N}(\mu,\Sigma)$, where $\mu\in\mathbb{R}^n$ and $\Sigma\in\mathbb{R}^{n\times n}$. How can I efficiently sample from $X | {\min{X}\le 0}$? (I.e. from the distribution of $X$ ...
4
votes
0answers
606 views

Gibbs sampling for spike and slab priors

In Spike and slab variable selection (equation 4) there is a model setup of the form $\beta_k | \lambda_k, \tau_k \sim \text{Normal} (0, \lambda_k \tau_k^2)$ $\lambda_k | \nu_0, w \sim (1-w)\delta_{\...
4
votes
0answers
485 views

Gibbs sampling for correlated random variables

Short summary Suppose two latent variables of a hierarchical model are correlated. Let $1-\epsilon$ be the degree of correlation. As $\epsilon\rightarrow 0$ the variables become perfectly correlated ...
4
votes
0answers
1k views

Gibbs sampling for LDA — does a small Dirichlet concentration parameter make a difference?

I'm using a Gibbs sampler for Latent Dirichlet allocation as described by Griffiths and Steyvers (http://www.ncbi.nlm.nih.gov/pmc/articles/PMC387300/). The sampling of a new topic $j$ for word $i$ is ...
4
votes
0answers
288 views

Convergence theorem for Gibbs sampling

The convergence theorem for Gibbs sampling states: Given a random vector $X$ with components $X_1,X_2,...X_K$ and the knowledge about the conditional distribution of $X_k$ we can find the actual ...
4
votes
0answers
4k views

Metropolis-Hastings within Gibbs sampling

Suppose we have the following classical normal linear regression model: $$y_i = \beta_1 x_{1i} + \beta_2x_{2i} + \beta_3x_{3i} + e_i$$ where $e_{i} \sim iid.N(0, \sigma^2)$ for all $i = 1, 2, \cdots,...
4
votes
0answers
295 views

Gibbs sampling from full conditionals

I have the following joint density: $p(x_1,x_2,y_1,y_2) \propto \exp\left(−\left(x_1^2+x_2^2+c_1(y_2-y_1)^2+c_2(y_2-y_1)^4\right)\right)$ Can I use Gibbs sampling to sample from that? How can I get ...
3
votes
2answers
2k views

Sampling variables and calculating likelihood in WinBUGS/OpenBUGS

I am trying to read some WinBUGS/OpenBUGS examples to figure out how to specify models. I can't seem to understand where the probabilistic dnorm, ...
3
votes
2answers
280 views

Gibbs sampling and Bayesian inference

I wanted to get a more in-depth understanding of sampling algorithms, and so I thought I could start with the very simple example of a binomial or bernoulli likelihood with a beta prior (since it is ...
3
votes
1answer
2k views

Gibbs sampling convergence

In an astronomical context, the authors of a paper desire to use a Gibbs algorithm. Please note: I am inexperience in MCMC algorithms, and specifically in Gibbs sampling. What we want, in essence, is ...
3
votes
1answer
571 views

Gibbs sampling an Ising model with 0s and 1s

One of my problems in one of my courses ask to sample a 20 dimensional vector of 0s and 1s, $\{0,1\}^{20},$ when they are distributed as $$ \pi(x) = \exp\left\{-\beta \sum_{i=1}^{19} |x_{i+1}-x_i| \...
3
votes
1answer
616 views

Gibbs sampling and Conjugate Priors

Are conjugate priors required when performing Gibbs sampling?
3
votes
1answer
261 views

Sampling a random binary matrix with “Gaussian” probability distribution

Let $A_{ij}$ be a $n\times n$ random binary matrix with probability mass function $P(A)$ given by $$ \log P(A)=-\frac 12 \mathrm{tr}\left[\left(A-M\right)^TV\left(A-M\right)\right] + C, $$ where $M$ ...
3
votes
1answer
185 views

What is a hierarchical model that can estimated via the Metropolis-Hastings Algorithm but not the Gibbs Sampler?

My understanding of the differences between MH and Gibbs Samplers is that a Gibbs Sampler is usually used when the full conditionals are present to us. In other words, it is a known distribution, so ...
3
votes
2answers
600 views

Markov Chain Monte Carlo (MCMC) with transformed data

I want to obtain an estimate of a parameter $\Theta$ in a model for a random variable $X$ dependent on $\Theta$ with known but complicated likelihood $L(\Theta|X) = p(X|\Theta)$. $X$ is not directly ...
3
votes
1answer
979 views

Gibbs sampling with mixed prior using a Metropolis-Hastings step

My questions are about a sampling procedure for fitting a Bayesian hierarchical model where one of the priors is a mixture distribution of discrete and continuous parts. The model is not my own but I ...
3
votes
1answer
505 views

Gibbs sampling with Dirichlet Likelihood

I have a sequence of observations that I am representing as proportions: X1 X2 X3 X4 X5 0.10 0.20 0.50 0.12 0.08 0.07 0.24 0.55 0.04 0.10 ... ...
3
votes
1answer
4k views

JAGS: posterior predictive distribution

I am fitting a simple linear regression model with R & JAGS: ...
3
votes
1answer
29 views

Slice Sampling asks to draw from $f^{-1}]y,+\infty[$

Slice Sampling asks to draw uniformly from $f^{-1}]y,+\infty[$. Wikipedia page However, how can we be sure that a uniform defined over the set $f^{-1}]y,+\infty[$ is in fact proper? If I had to ...
3
votes
1answer
1k views

Gibbs measure and normal distribution

On Wikipedia, the Gibbs measure defines the probability as: $$ P(X=x) = \frac{1}{Z(\beta)}\exp(-\beta E(x)) $$ Now, the familiar form of the normal distribution is: $$ P(x) = \frac{1}{\sqrt{2\pi}\...
3
votes
1answer
782 views

Prior selection for Gaussian Processes (GP)

I am trying to select a prior for the covariance parameters of my Gaussian Process (GP) and have been running into numerical problems with my MCMC code. My model is the following: $$Y = D\beta + GP(...
3
votes
1answer
2k views

Sampling from conditional distribution in general case

I'm dealing with Gibbs Sampling now. Let's consider the example: I know the distribution of X|Y and the distribution of Y. They are some known - Binomial or Beta or other but particular. Thus I have ...