The Gibbs sampler is a simple form of Markov Chain Monte Carlo simulation, widely used in Bayesian statistics, based on sampling from full conditional distributions for each variable.

learn more… | top users | synonyms

1
vote
1answer
18 views

How to incorporate parameter constraints in Metropolis Hastings

I am working on parameter estimation of GARCH model with Metropolis Hastings. But the results I have got doesn't look reasonable, actually it is quite different from what I have got from Gibbs ...
1
vote
1answer
26 views

Gibbs Sampler - Sample mean convergence

To simulate from the posterior distribution $p(\theta|Y)$ where $\theta = (\mu,\lambda_1,\lambda_2)$, I run a Gibbs sampler to draw approximately random values from $p(\theta|Y)$. This Gibbs sampler ...
4
votes
1answer
26 views

Gibbs Sampler output: how many Markov chains?

When running a Gibbs sampler (for $n=200$ Iterations) with two full conditionals, I get the output $\mathbf{x} = (x_1^{(n)},x_2^{(n)})_{n =1,...,200}$. So $\mathbf{x}$ is the realizations of a Gibbs ...
0
votes
0answers
7 views

generating a sequence of indicators based on variable boundaries in JAGS

Suppose I have a vector of indices 1:N, and data y[1], ..., y[N]. I have three variable center points in ...
2
votes
1answer
30 views

Full conditionals - Gibbs Sampler

i want to draw samples from a 5-dimensional posterior distribution $f(k,\theta,\lambda,b_1,b_2|Y=y)$. From Bayes-Theorem there is the following relationship between posterior and likelihood: ...
2
votes
1answer
35 views

Implementing Gibbs sampler in R from posterior distribution

I am referencing a follow-up idea from something I posted earlier (Zero-inflated Poisson and Gibbs sampling, proofs and sampling). I want to implement the Gibbs sampler, by generating a large ...
3
votes
1answer
27 views

Zero-inflated Poisson and Gibbs sampling, proofs and sampling

I am trying to figure out zip-inflated Poisson (ZIP) model. In this model, random data $X_1, .., X_n$ are of the form $X_i=R_iY_i$, where the $Y_i$'s have Poisson distribution ($\lambda$) and the ...
1
vote
0answers
18 views

Bayesian estimates for Deming regression coinciding with least-squares estimates

Consider the following Deming model with independent replicates : $$x_{i,j} \mid \theta_{i} \sim {\cal N}(\theta_{i}, \gamma_X^2), \quad y_{i,j} \mid \theta_{i} \sim {\cal N}(\alpha+\beta\theta_{i}, ...
0
votes
1answer
50 views

Gibbs Sampler Running Wild

So, I'm setting up a Gibbs Sampler using a multivariate normal model with a Jeffreys prior (working through the Hoff book on my own). There's also missing data to be imputed. I've checked my posterior ...
1
vote
0answers
12 views

Gibbs sampling for concentration parameter in Dirichlet Process Mixture models

Let's assume we have a DP mixture model: \begin{align} G &\sim {\rm DP}({\alpha, H})\\ \theta_i &\sim G \\ x_i &\sim F(\theta_i) \end{align} There are many methods to find the posterior ...
2
votes
2answers
97 views

MCMC chain getting stuck

I am trying to use a Metropolis-within-Gibbs type algorithm to sample $\theta$ and $x$ from the following model. Starting with Bayes theorem I can write: $$ P(\theta, x | y) = \frac{P(y | x, \theta) ...
2
votes
1answer
53 views

Derive the Gibbs sampler for this bivariate distribution

I understand the theory of Gibbs sampling. It is an iterative sampling algorithm that defines, sequence of random variables with the property of a Markov chain. Specifically, I choose any starting ...
0
votes
0answers
22 views

How to include a half-Cauchy prior in a Gibbs sampler for a hierarchical model (Rubin's “8 schools”)?

In the third edition of the popular Bayesian Data Analysis, Gelman et al. discuss a variation on Rubin's "8 schools" problem in which only three schools are considered (p. 131). The authors suggest ...
0
votes
1answer
36 views

PyMC consistently under estimating results found in paper. Possibly not sampling enough?

I have been trying to build confidence in (my ability to correctly use) PyMC by working examples. Namely, I have been working on Chickering and Pearl 1997, and more specifically on their 'artificial' ...
2
votes
1answer
87 views

Confusion in Gibbs sampling

I am self-studying Gibbs sampling from a book. The book introduces metropolis hastings algortihm to generate representative values from a posterior distribution. So we know $p(D | \theta) p(\theta)$ ...
4
votes
1answer
22 views

Why does Slice sampler use the log of the density?

The Slice sampler1 takes as its argument the log of the density to be sampled from. Why is it doing this? A commenter on this question pointed out that it makes no sense to "sample" from the log of ...
0
votes
1answer
55 views

Metropolis-Hastings using log of the density

Does Metropolis-Hastings work with the log of the proposal and the density to be sampled from? That is, say we want to sample from a density $\pi(x)$, using a proposal $q(x|x^{old})$, will the ...
0
votes
0answers
12 views

General state space

is there a clear definition for a "general state space" in the sense of Markovchains ? Is for example $\mathbb{N}$ a general state space because it is countable infinity?
1
vote
1answer
96 views

Clustering methods for unknown number of clusters

Matrix $X=[x_1,...,x_i,...,x_N]$ is a data-set containing $N$ data-points that each data-point $x_i$ is a vector of $D$ dimensions. Each dimension is a feature. The number of clusters ($K$) is ...
0
votes
0answers
15 views

Suggestions of a statistical model with IID data and latent variables

While this might be an unusual request, I am looking for a statistical model with certain properties to test my numerical method on and thought I might ask here. The model ought to have the following ...
4
votes
1answer
88 views

Gibbs Sampler transition kernel

Let $\pi$ be the target distribution on $(\mathbb{R}^d,\mathcal{B}(\mathbb{R^d}))$ which is absolutely continuously wrt to the $d$-dimensional Lebesgue measure, i.e : $\pi$ admits a density ...
5
votes
1answer
182 views

Gibbs Sampler contradiction proof

I want to prove that the systematic scan Gibbs sampler yields an aperiodic chain $X$ on a general state space. Let $\pi$ be the stationary distribution for the resulting chain. Suppose to get a ...
1
vote
0answers
15 views

Efficient generation of graph structured correlated random variables via MCMC/Gibbs

Sometime back I had asked this question about generating correlated random draws based on the correlation structure given by a graph. Link Here The solution there requires to create $n\times n$ ...
1
vote
1answer
64 views

JAGS, cannot evaluate upper index of counter

I asked this question at the JAGS sourceforge help forum but didn't get response there. I have the following JAGS model: ...
1
vote
0answers
39 views

How can I sample multivariate binary variables such that sum of them follows a gamma distribution?

Edit: Since the original question was confusing as whuber pointed out, let me rephrase the question with a Poisson distribution instead of a gamma distribution. The energy term of a Poisson ...
2
votes
2answers
426 views

Metropolis-Hastings Algorithm within Gibbs Sampling

I have this $f$ function below. $$ f(x_1,x_2)\propto \left(\dfrac{x_1}{x_2}\right)\left(\dfrac{\alpha}{x_2}\right)^{x_1-1}exp\left\{-\left(\dfrac{\alpha}{x_2}\right)^{x_1} \right\}I_{R^+}(x) $$ where ...
1
vote
1answer
37 views

MCMC sampling with sum constraints

I'm interested in sampling a collection of variables with a sum constraint on them. For a simplified example: Prior: $X \sim \mathcal{N}(0, 1)$ $Y \sim \mathcal{N}(0, 1)$ Observation: $X + Y = 1$ ...
1
vote
0answers
34 views

Gibbs sampling version for estimating the Dynamic Topic Model (DTM)?

The paper of Blei et Lafferty published at ICML'06 implements a (quite complicated) variational inference (VI) technique for estimating the parameters of the Dynamic Topic Model, see: ...
0
votes
1answer
36 views

Gibbs Sampling for Boltzmann Machines

David Mac Kay, in his book on machine learning talks about Boltzmann machines, and on pg. 3 here http://www.inference.phy.cam.ac.uk/itprnn/ps/521.526.pdf He says "the second equation ...
1
vote
1answer
66 views

When should we use Gibbs Sampling in a deep belief network? Before or after fine-tuning?

Gibbs sampling allows for sampling a vector with a deep belief network. There are two steps to training a DBN for a supervised learning task: greedy unsupervised pre-training and supervised ...
5
votes
1answer
63 views

How to derive Gibbs sampling?

I'm actually hesitating to ask this, because I'm afraid I will be referred to other questions or Wikipedia on Gibbs sampling, but I don't have the feeling that they describe what's at hand. Given a ...
8
votes
1answer
222 views

Marginal Likelihood from the Gibbs Output

I'm reproducing from scratch the results in Section 4.2.1 of Marginal Likelihood from the Gibbs Output Siddhartha Chib Journal of the American Statistical Association, Vol. 90, No. 432. (Dec., ...
0
votes
0answers
14 views

What sort of data would be appropriate to analyze under an MCMC method?

MCMC methods describe stochastic sampling but I'm not entirely sure the contexts in real datasets one would wish to apply MCMC methods. What kind of data could I gain insight into with MCMC methods?
2
votes
0answers
24 views

Comparison of MCMC methods? [closed]

Where can I find a good comparison of Gibbs, Metropolis, and Hybrid MCMC in R or Python? I have thus far found this ...
2
votes
1answer
74 views

Need help deriving a gibbs sampler for a normal mixture model with two components

Let $\theta_i$ be an indicator that the i-th eruption is a long eruption. (i.e. $\theta_i = 1$ if the i-th eruption is long and $\theta_i = 0$ otherwise.) Assume the following model and derive a ...
0
votes
0answers
25 views

Gibbs Sampling with given Posterior Distribution

I'm trying to implement an algorithm from a paper which assigns three types of labels $m_i, m_d$ and $m_s$. Here $m_i$ labels a collection of documents $G_i , m_d$ a subcollection of them and $m_s$ ...
1
vote
2answers
36 views

Sampling from a portion of the normal distribution?

I have a a conditional distribution $p(X_1 | \theta) \propto MVN(\mu, \Omega) \pi(X_1)$ where $X_1=[x_1, x_2, \dots, x_n]'$ and $\pi(X_1)=1$ when all $x_i \in [0,a)$ and $0$ otherwise. Is there any ...
4
votes
2answers
76 views

Gibbs Sampling and Probability Notation

Problem 1 I am trying to implement Gibbs Sampling for the following problem: There is a grid measuring 3 x 3 sites, each "site" can be designated in a state, $X$, of 1 or -1. The sites are numbered ...
0
votes
0answers
26 views

Gibbs within Metropolis

Consider a model with two parameters, $\alpha$ and $\beta$. We want to sample these two parameters conditioning on two data points, $d_1$ and $d_2$. Is it possible to use an algorithm like this: 1) ...
4
votes
1answer
67 views

Methods of fitting a dynamic linear model

I'm taking a time series course and am learning about exchangeable time series form of dynamic linear models (DLMs). This is given by: \begin{align*} \mathbf{y}_t' &= ...
0
votes
0answers
24 views

Gibbs Sampling form Non-standard distribution in R

Gibbs sampling examples in R has involved initializing and updating iteratively from conditional that are in standard form. Has anyone performed on a gibbs sampling when the conditionals are in ...
6
votes
1answer
234 views

Does the Gibbs Sampling algorithm guarantee detailed balance?

I have it on supreme authority1 that Gibbs Sampling is a special case of the Metropolis-Hastings algorithm for Markov Chain Monte Carlo sampling. The MH algorithm always gives a transition probability ...
3
votes
0answers
108 views

Is my OpenBUGS / WinBUGS model well specified?

I've just started trying to use OpenBUGS for Bayesian analysis of stochastic volatility models. In particular, I'm trying to calculate stochastic covariance, similar to the DC-MSV model specified by ...
10
votes
1answer
132 views

Bayesian modeling using multivariate normal with covariate

Suppose you have an explanatory variable ${\bf{X}} = \left(X(s_{1}),\ldots,X(s_{n})\right)$ where $s$ represents a given coordinate. You also have a response variable ${\bf{Y}} = ...
2
votes
1answer
43 views

Posterior parameter distribution

I am considering the following non-linear state space model: $X_t=\frac{X_{t-1}}{2}+25\frac{X_{t-1}}{1+X_{t-1}^2}+8\cos{1.2t}+\epsilon_t, \epsilon_t\sim N(0,\sigma_x^2 ) $ ...
1
vote
0answers
67 views

Understanding a measure of convergence of MCMC simulations

I am trying to better understand better the Gelman/Rubin measure of convergence of MCMCs. The method starts off by defining two quantities: $B$ and $W$. $B$ is said to be the between chain variance ...
0
votes
0answers
13 views

STEP BY STEP approach to learn LDA (Latent Dirchlet Allocation)

I have an urgent need to understand Latent Dirchlet Allocation (LDA) for Topic Modeling. I tried several sources, but it seems I do not have required knowledge to learn this. My statistics knowledge ...
2
votes
1answer
64 views

Is Slice Sampling a special case of Gibbs Sampling?

I read on this thread the following: If you can use both the gibbs sampler and slice sampling to sample from a posterior I would use the Gibbs sampler as the slice sampler seems unnecessary to ...
1
vote
1answer
140 views

Gibbs Sampling Detecting Change point in time series

I was reading through this one page paper on using Gibbs sampling for detecting a change point in a time series like data. While I understand the part where the $\lambda$ and $\phi$ are chosen from a ...
0
votes
0answers
10 views

References on deterministc augmented variable for Gibbs sampling

According to Wikipedia, It is also possible to extend Gibbs sampling in various ways. ... It is ... possible to incorporate variables that are not random variables, but whose value is ...