Questions tagged [gibbs]

The Gibbs sampler is a simple form of Markov Chain Monte Carlo simulation, widely used in Bayesian statistics, based on sampling from full conditional distributions for each variable or group of variables. The name comes from the method being first used on Gibbs random fields modeling of images by Geman and Geman (1984).

Filter by
Sorted by
Tagged with
-1
votes
0answers
17 views

Estimate parameters with observation sequence using Gibbs sampling [closed]

Due to limited understanding about MCMC theory, unable to figure out how to use it to resolve a specific question as below: The result space consists of 5 discrete possible values, which satisfy the ...
0
votes
0answers
6 views

Inference on Author model

The Author Model is an LDA based model that first time introduced in paper [The Author-Topic Model for Authors and Documents]. I have studied the inference of the LDA model and know how to obtain the ...
1
vote
0answers
15 views

parameter estimation on the LDA model

I have a problem with estimating the parameters of $\theta$, and $\phi$ in the Latent Dirichlet Allocation (LDA) model. The article Finding scientific topics has done the estimation of the parameters ...
1
vote
1answer
30 views

Relationship between variational inference and sampling in a Boltmzann-machine-like network

In this paper concerning a Boltzmann-machine-like network and its variational mean field approximation, the authors write In the stochastic system as well as the deterministic system, units evolve ...
1
vote
0answers
78 views

Sampling from conditional distribution for Gibbs sampler

I want to implement a gibbs sampler to help fit some parameters to a model. I have a kolmogorov-forward equation, which I can solve (at steady state) to give me $P(x|\theta)$, the probability of ...
0
votes
0answers
19 views

What pitfalls should we avoid with Heidelberger-Welch convergence

I'm working through validating a Bayesian mixture model for multi-species occupancy with a collaborator. Initially, we relied on coda::heidel.diag to alert us to ...
1
vote
1answer
35 views

Multi-steps direct forecasting in AR(2) model through bayesian estimation of the model

I'm estimating an AR(2) model using Bayesian methods through Gibbs sampling and I want to perform 4 step ahead multi-steps direct forecasts. Inside the MCMC loop in each iteration I'm drawing the ...
-1
votes
1answer
20 views

Which sampling methods (MCMC or otherwise) can be used if the posterior distribution is unknown?

The goal is to sample the posterior distribution of parameters describing some model (fairly low dimensional, generally no more than 10 parameters at the absolute most, usually around 5), but I don't ...
0
votes
0answers
13 views

How to calculate marginal transition matrix given the joint probability distribution for Gibbs sampling?

If I have two variables x$_{1}$ and x$_{2}$ which both take values in {1,2,3}, and I have the table representing their joint probabilities, how can I then determine the transition matrix for the ...
0
votes
1answer
26 views

Replicating an experiment on GMRF (Gaussian Markov Random Field)

I am trying to understand an experiment from this paper, specifically Section 5.2. In the paper, they propose a new algorithm for computing the log-determinant of sparse matrices, and in section 5 ...
2
votes
1answer
59 views

What should be the burn in period for Metropolis-within-Gibbs?

I need to get samples from an unnormalized distribution $p(\theta, \tau | D)$. However, sampling directly from the joint distribution with Metropolis-Hastings is hard, as the sampler rarely finds ...
1
vote
0answers
19 views

Gibbs sampler with adaptive linear transformation

It is a well known fact that linear transformations can dramatically improve the performances of a Gibbs sampler when a ridge-like joint likelihood function occurs. Can I make an algorithm that ...
0
votes
0answers
15 views

Why does full conditional distribution and posterior distribution have the same distribution? [duplicate]

I just started to study Bayesian statistics and there is something confusing me. I need a full conditional distribution if I want to sample by means of Gibbs Sampler. Also I found that the full ...
0
votes
0answers
32 views

Building an hierarchical model

I am trying to understand hierarchic Bayesian models. I am following the example http://www.openbugs.net/Examples/Pumps.html. The exercise I am trying to complete relates to the same problem, with ...
0
votes
0answers
26 views

Sampling from hyperprior with Gibbs sampling

Let's say I have some set of data, $\vec{y}$, where each element is sampled as $$ y_n \sim Normal(\mu,1/\tau) $$ where $\tau$ is the precision, $1/\sigma^2$, and I want to use Gibbs sampling, choosing ...
1
vote
1answer
37 views

Gibbs sampling example of a bivariate normal with unknown correlation

I'm looking for an example of using Gibbs sampling with a bivariate normal, where the correlation parameter is not fixed or known. In other words, what is the conditional distribution of the ...
1
vote
1answer
38 views

Gibbs sampling for Multivariate: how to update?

In this page of Murphy's 'Machine Learning: a Probabilistic Perspective' it's explained how to do Gibbs sampling on a Gaussian Mixture Model. Reading this, I was trying to understand when to update ...
3
votes
2answers
49 views

R alternatives to JAGS/BUGS [closed]

I've recently fit more complex hidden markov models with random effects and covariates etc. JAGS was the only program that could get the job done. Now I want to write my own functions to facilitate ...
1
vote
1answer
61 views

Stationarity of coefficients when sampling VAR by Gibbs sampler

I am using Gibbs Sampler for VAR and have noticed that some researchers check the stationarity of $\beta$ coefficients while drawing. I am not sure why do they do that? Bayesian VARs do not require ...
0
votes
0answers
42 views

Derivation of a Gibbs sampler for a Bayesian model with hierarchical Dirichlet prior

I am studying Gibbs sampling. In particular, I got stuck on deriving Gibbs sampling when the reference Bayesian model has hierarchical Dirichlet distributions. As an example, let us start with the ...
1
vote
1answer
40 views

Gibbs Sampling - Calculating the full conditionals from the joint density

Given a joint density, $f(x_1, x_2)$, can its pmf/pdf be found generally by the method outlined below: For a joint density, $f(x_1, x_2)$ if we hold $x_2$ constant in the joint density, we will get ...
3
votes
1answer
78 views

Do I need to evaluate acceptance rates in Metropolis within Gibbs algorithm?

Consider the Gibbs sampler Sample $\theta' \sim p(\theta|\tau, D)$ Sample $\tau' \sim p(\tau|\theta', D)$ where $\theta,\tau$ parameters of the data $D$. Now assume that we can only sample from $p(\...
1
vote
0answers
31 views

Sampling states of an “unnatural” Hamiltonian System

I would like to sample from a Gibbs distribution given by $$f(p, q) = \frac{1}{\mathcal{Z}}e^{-H(p, q; \omega, J)}$$ where $H$ is the Hamiltonian on generalized coordinates $(p,q)\in \mathbb{R}^{2n}...
1
vote
0answers
21 views

Why the nodes in a Boltzmann machine need to be sampled one at a time?

Typically, we use Gibbs sampling to update (or generate samples from) energy based models. This means we update each node while keeping its markov blanket constant. Why can't we update/sample all ...
0
votes
1answer
90 views

What does MCMC do during burn-in period?

I am studying mcmc and I am wondering what mcmc does during burn-in period. And also what is the difference during burning period and after the burn-in period?
1
vote
1answer
69 views

What is the role of simulated annealing in Gibbs sampling?

While I was reading about Gibbs sampling, I happened to see "simulated annealing" but what is it doing in Gibbs sampling? Although I don't understand the full context of simulated annealing, I am ...
0
votes
0answers
25 views

Does MCMC Gibbs sampling algorithm first build a steady Markov Chain, then does the sampling to build the posterior distribution?

I am currently studying MCMC Gibbs sampling and while reading this part, a question has come into my head if MCMC Gibbs sampling first build a steady Markov Chain and does the sampling or does ...
2
votes
2answers
63 views

Need to understand a statement for Random Walk Metropolis algorithm's proposal distribution?

I was told that the proposal distribution of Random Walk Metropolis needs to be symmetric. But today I was reading a book about Bayesian Analysis which contains the following statement: "The proposal ...
1
vote
0answers
83 views

Bayesian Gamma Regression Update

I'm looking for a resource that explains how to do update the coefficients for a Bayesian gamma regression using Gibbs sampling. Specifically, if $y_i \sim Gamma(\alpha,\beta_i)$ and my data ...
0
votes
0answers
28 views

Gibbs sampling for simple posterior distribution?

I have a likelihood function, $$ p(x) = \theta^{\sum x} (1- \theta)^{n-\sum x} $$ and prior distribution, $$ p(\theta) \propto \theta^{\alpha - 1} (1- \theta)^{\beta - 1}$$ then the posterior ...
0
votes
0answers
15 views

Belief Propagation vs Gibbs Sampling

In general what are the cons and pros of using Gibbs sampling to estimate a complex posterior (assuming we can sample from the conditionals) over belief propagation (using a factor graph)?
1
vote
0answers
19 views

Markov chain Monte Carlo, Mixing Time

How do you estimate the mixing time for a markov chain? I read somewhere one can use the sum of the auto-correlation coefficients or the sum of the auto-covariance coefficients, but I cannot seem to ...
0
votes
0answers
21 views

sampling question in Gibbs sampling for a Gaussian mixture model

I have some confusions regarding the Gibbs sampling step for the following mixture model: consider a mixture model of the following generative process: $\theta \sim Dir(\alpha) $ (global hidden ...
1
vote
0answers
46 views

What's the point of Gibbs Sampling? [duplicate]

I am reading a book on doing Bayesian Data Analysis. I have just learned what the Metropolis Hastings (MH) Algo does, at least in relation to Bayesian Data Analysis. My understanding of the MH Algo ...
0
votes
0answers
19 views

the convergence speed for a Markov chain

For a metropolis hastings algorithm, suppose that the stationary distribution is defined as the Gibbs Boltzmann distribution $\pi_T(x)= \frac{1}{Z_T}e^{-\frac{V(x)}{T} }$ where $Z_T = \sum_{y\in V} e^{...
0
votes
0answers
18 views

Dirichlet Process mixture model with independent features

I'm trying to construct a Dirichlet process mixture model for clustering where the samples have independent features. In other words, to evaluate the likelihood of sample $x_i$, I would compute $\...
0
votes
1answer
21 views

Gibb's sampling where target prob distribution is itself a conditional joint distribution - p(x,y|t)

I'm new to Gibb's sampling and need basic guidance. Say p1,p2,q-> are Gaussian variables. p1->q<-p2 and q->x where x is a discrete variable (either 1 or 0). How do I go about Sampling (using ...
1
vote
1answer
25 views

Assign an error to the parameters of MAP estimate

Through a MCMC Gibbs sampler I obtain $M$ chains of the parameters vector $\mathbf{\theta}$, meaning that each component of $\mathbf{\theta}$ is the value of one parameter at a given iteration. ...
0
votes
1answer
127 views

GIBBS sampling : do samples for the subset of variables approxiamate the related marginal distribution?

I'm reading the page Gibbs sampling on wikipedia. I really don't understand why the following statement is true. "The marginal distribution of any subset of variables can be approximated by simply ...
1
vote
0answers
27 views

Is there something wrong with my Bayesian hierarchical spatio-temporal model?

I built a Bayesian spatio-temporal model and one of the parameters to be estimated is the random spatial effects s. The random spatial effect is assigned an intrinsic conditional autoregressive prior (...
4
votes
1answer
87 views

Is the joint distribution $P_{XY}(x,y)$ determined from the conditionals $P_{X|Y}(x|y)$ and $P_{Y|X}(y|x)$?

For simplicity assume that $X,Y$ are discrete, finite, random variables, with joint distribution $P_{XY}(x,y) = \mathbb{P}(X=x\wedge Y=y)$. Now suppose that we do not know $P_{XY}(x,y)$, but are ...
0
votes
0answers
37 views

sampling from joint distribution to recover marginal distribution

I'm going through Bayesian Core and have gotten stuck at this remark on page 233: " A first remark that motivates the use of the Gibbs sampler is that, within structures such as $$ \pi(x_1) = \int \...
0
votes
1answer
41 views

Gibbs sampling: Conjugate prior for a two component known-unknown variance?

If I have a normally distributed variable $\mathcal{N}(\mu,\frac{1}{\tau})$ with fixed $\mu$ then the conjugate prior for an unknown $\tau$ is then $\mathcal{Ga}(\frac{n}{2}+\alpha, \beta + \sum_i \...
0
votes
0answers
18 views

Any relationship between MAP full conditionals and MAP joint?

Context: Bayesian model that one can draw posterior samples from via Gibbs sampling of the relevant full conditionals. Question: Can anything be said (i.e. bounds, conditions for equality) about the ...
0
votes
0answers
14 views

Gibbs Sampling attempt at a simple Coxian distribution

I have the following Coxian model for inter-arival times ($x_i$) that has $C_x^2 < 1$: $$ p(x_i\mid \lambda,\theta) = \theta \lambda^r x_i^r e^{-\lambda x_i} + (1-\theta)\lambda e^{-\lambda x_i} $...
1
vote
0answers
45 views

Understand the Holmes and Held (2006) Bayesian probit MCMC algorithm

Holmes and Held (2006) suggest a simple approach to reduce autocorrelation in the MCMC algorithm proposed by Albert and Chib (1993). HH (2006) propose to update $\beta$ and $z$ jointly, making use of ...
0
votes
0answers
96 views

Gibbs Sampler for GMM

In Rasmussen's paper it is introduced a Gibbs sampler to make inference about a standard Gaussian Mixture Model. To simplify, assume the 1-d case with basic hierarchical structure, that is: $x_i|...
2
votes
0answers
30 views

Gibbs sampling for mixture with Dirichlet prior?

I want to sample from the distribution of a mixture distribution. The hierarchical model is $x_i\sim f$, where: $$f(x\mid \theta_1,\dots,\theta_p, w_1,\dots,\omega_p) = \sum_{j=1}w_p\varphi(x\mid\...
0
votes
0answers
33 views

Shape of parameters marginal posterior in hierarchical Bayes model

Consider a generic hierarchical Bayes model with data $y_i\sim p(y|\theta_i)$, dependent of parameters $\theta_i\sim p(\theta|\phi)$ and hyperparameters $\phi\sim p(\phi)$. Furthermore, assume that $\...
3
votes
1answer
64 views

Auxiliary variable Gibbs sampler

Suppose we want to sample from a pdf $f(x_1,x_2)$. It's easy to sample from $x_1 \vert x_2$, but not $x_2 \vert x_1$, so we introduce an auxiliary variable $u$ such that $\int f(x_1,x_2,u) du = f(x_1,...

1
2 3 4 5
8