Questions tagged [gibbs]

The Gibbs sampler is a simple form of Markov Chain Monte Carlo simulation, widely used in Bayesian statistics, based on sampling from full conditional distributions for each variable or group of variables. The name comes from the method being first used on Gibbs random fields modeling of images by Geman and Geman (1984).

Filter by
Sorted by
Tagged with
1
vote
1answer
35 views

Why iterations of Gibbs sampling for a bivariate Gaussian distribution can be seen as random walk?

In Section 4.4 of the excellent technical report Probabilistic Inference using Markov Chain Monte Carlo Methods, the author tries to analyze the performance of Gibbs and Metropolis algorithm with ...
1
vote
0answers
16 views

Plotting a random walk on R [closed]

I've run a Gibbs sampler and obtained a sample for $X_1$ and $X_2$. I'm trying to recreate a plot like this one: How do I recreate the walk part on R?
0
votes
0answers
8 views

How to sample posterior distribution for models with random effects?

I have a time series model contains some fixed parameters ($\beta_{1}$, k, m, etc. ) and also a random effect (i.e. $\beta_{t}$ follows a random walk, with starting value $\beta_{1}$ and variance ...
0
votes
0answers
19 views

How to improve mixing and effective size using Gibb's Sampling?

I have some data and I am trying to fit a Bayesian model. There are two things that I am unsure of: The mixing of my trace plots The effective size So I know that I can improve mixing by increasing ...
0
votes
1answer
70 views

How to built Gibbs sampler of Mixture Bayesian regression in R?

I am working on a Gibbs sampler of three parameters and we know the full conditional distribution of three parameters.
1
vote
0answers
20 views

Particle Gibbs Sampler For Regime-Switching Nonlinear Gaussian SSM

I'm reading this paper on using a non-linear Gaussian SSM for measuring regime-switching leverage effect using stock market data. I'm using it as jump-off point for an undergraduate paper. My advisor ...
2
votes
0answers
21 views

Gibbs updating algorithm (Gibbs steps) for computationally expensive likelihood

I am looking for a good way to update steps in a Gibbs sampler where the likelihood function is computationally expensive. Here is what I tried so far: By default JAGS uses a slice sampler. However, ...
0
votes
0answers
24 views

Deriving a full conditional distribution when Half-t distribution is used

I'm trying to use a Half-t distribution with Gibbs sampling to form a model. but having hard time finding the full conditional distribution. Suppose $$Y = X \beta + \epsilon $$, where $Y$ is a ...
1
vote
0answers
12 views

Gibbs sampling Bayesian conditional distribution for mean of a Normal distribution

first post here in CV. I'm currently working on a textbook exercise on Gibbs Sampling and got stuck on naming the distribution for one of the conditional distributions. Question Consider a normal ...
2
votes
0answers
40 views

Gibbs sampler of a generative model

I understand what a Gibbs sampler is and I understand how LDA does classification. But I'm unsure how I can generate a Gibbs sampler for an LDA model and how to meld the two concepts. Let's say I ...
4
votes
1answer
73 views

Bayesian mixture model joint posterior

I am just starting to learn about bayesian mixture models. There is a few clarifications that I want to make which I am not sure myself. The graphical model below describes a gaussian mixture model ...
3
votes
1answer
75 views

Implementation of a blocked Gibbs sampler for a mixture model with a Dirichlet-process prior

I am trying to understand and implement the blocked Gibbs sampler described on page 552 in Bayesian Data Analysis by Gelman et al. in the context of using a Dirichlet process as a prior in a mixture ...
1
vote
2answers
138 views

Deriving full conditionals from joint distributions?

In this link (https://www.youtube.com/watch?v=a_08GKWHFWo), the author derives the conditional distributions from the joint; but I got lost in the mechanics of what happened, the process was overly ...
1
vote
0answers
40 views

Gibbs sampling proposals for bivariate normal?

I'm very familiar with Metropolis-Hastings, having implemented the algorithm myself to handle "toy problems." Gibbs sampling, however, is a bit trickier for me as I'm not quite certain what ...
2
votes
0answers
44 views

Intuition on why Gibbs Sampling samples from the posterior distribution

I am new to Gibbs Sampling and I do understand how the algorithm works but I would also like to understand how sampling from the conditional distributions is equivalent to sampling from the joint. ...
0
votes
0answers
12 views

Uncorrelated Samples from a non-conjugate (but well behaved) posterior

I'm trying to create a Dirichlet process mixture model with a kernel distribution similar to a product of gammas. (in fact, if I generate a latent random variable, it IS a product of (independent) ...
1
vote
0answers
18 views

How does pymc3 posterior simulation work in this simple case without having the full conditional distributions?

I'm trying to estimate the posterior distribution of the gamma parameters alpha and beta given that my data comes from a gamma distribution and the priors I chose come from two uniform distributions. ...
0
votes
1answer
33 views

Running several MCMC chains after convergence?

I am running a MCMC Gibbs sampler for a computationally expensive model. It takes ~12 hours to obtain 1000 iterations of this MCMC sampler. I have tested the sampler, and I found that the chain seems ...
0
votes
0answers
37 views

Estimating parameters with Gibbs sampling?

I've been trying to understand Gibbs sampling; my end goal is to intuitively understand it in the context of MCMC methods. However, in order to reach that end, I started a with simpler example. I ...
0
votes
0answers
14 views

Stationary distibution of Gibbs sampler

I am suppose to find stationary distribution of a Marcov chain generated by the following Gibbs sampler $n := 0$; $X_0 = x_0$; $(x_0>0)$ repeat Gen $Y_n \sim U(0, \exp(-X_n))$; Gen $X_{n+1} \...
0
votes
0answers
61 views

python gibbs sampler for bivariate normal distribution, failing to converge

I've been trying to understand Gibbs sampling for some time. Recently, I saw a video that made a good deal of sense. https://www.youtube.com/watch?v=a_08GKWHFWo The author used Gibbs sampling to ...
4
votes
2answers
141 views

What is the relationship between Boltzmann / Gibbs sampling and the softmax function?

I'm looking at sampling functions in the context of reinforcement learning; specifically the explore/exploit problem. A method I've seen pretty often is to derive the action by assigning a score to ...
0
votes
1answer
41 views

Why does my Gibbs sampler find two optimals?

[EDITED] I am using a Gibbs Sampler to find a Bayesian optimization to my multilevel (hierarchical) model (2 levels). However, when I run multiple chains (each chain having different starting values) ...
0
votes
1answer
41 views

Gibbs Sampler Conditional Marginal Computation

I have the following question regarding the Gibbs sampler, although it might be considered a simple question on conditional probability. For sake of simplicity, let us say we are trying to sample ...
0
votes
0answers
8 views

Inference on Author model

The Author Model is an LDA based model that first time introduced in paper [The Author-Topic Model for Authors and Documents]. I have studied the inference of the LDA model and know how to obtain the ...
1
vote
0answers
37 views

parameter estimation on the LDA model

I have a problem with estimating the parameters of $\theta$, and $\phi$ in the Latent Dirichlet Allocation (LDA) model. The article Finding scientific topics has done the estimation of the parameters ...
1
vote
1answer
43 views

Relationship between variational inference and sampling in a Boltmzann-machine-like network

In this paper concerning a Boltzmann-machine-like network and its variational mean field approximation, the authors write In the stochastic system as well as the deterministic system, units evolve ...
0
votes
0answers
42 views

What pitfalls should we avoid with Heidelberger-Welch convergence

I'm working through validating a Bayesian mixture model for multi-species occupancy with a collaborator. Initially, we relied on coda::heidel.diag to alert us to ...
1
vote
1answer
57 views

Multi-steps direct forecasting in AR(2) model through bayesian estimation of the model

I'm estimating an AR(2) model using Bayesian methods through Gibbs sampling and I want to perform 4 step ahead multi-steps direct forecasts. Inside the MCMC loop in each iteration I'm drawing the ...
-1
votes
1answer
39 views

Which sampling methods (MCMC or otherwise) can be used if the posterior distribution is unknown?

The goal is to sample the posterior distribution of parameters describing some model (fairly low dimensional, generally no more than 10 parameters at the absolute most, usually around 5), but I don't ...
0
votes
0answers
30 views

How to calculate marginal transition matrix given the joint probability distribution for Gibbs sampling?

If I have two variables x$_{1}$ and x$_{2}$ which both take values in {1,2,3}, and I have the table representing their joint probabilities, how can I then determine the transition matrix for the ...
0
votes
1answer
123 views

Replicating an experiment on GMRF (Gaussian Markov Random Field)

I am trying to understand an experiment from this paper, specifically Section 5.2. In the paper, they propose a new algorithm for computing the log-determinant of sparse matrices, and in section 5 ...
2
votes
1answer
174 views

What should be the burn in period for Metropolis-within-Gibbs?

I need to get samples from an unnormalized distribution $p(\theta, \tau | D)$. However, sampling directly from the joint distribution with Metropolis-Hastings is hard, as the sampler rarely finds ...
1
vote
0answers
19 views

Gibbs sampler with adaptive linear transformation

It is a well known fact that linear transformations can dramatically improve the performances of a Gibbs sampler when a ridge-like joint likelihood function occurs. Can I make an algorithm that ...
0
votes
0answers
36 views

Building an hierarchical model

I am trying to understand hierarchic Bayesian models. I am following the example http://www.openbugs.net/Examples/Pumps.html. The exercise I am trying to complete relates to the same problem, with the ...
0
votes
0answers
41 views

Sampling from hyperprior with Gibbs sampling

Let's say I have some set of data, $\vec{y}$, where each element is sampled as $$ y_n \sim Normal(\mu,1/\tau) $$ where $\tau$ is the precision, $1/\sigma^2$, and I want to use Gibbs sampling, choosing ...
1
vote
1answer
54 views

Gibbs sampling example of a bivariate normal with unknown correlation

I'm looking for an example of using Gibbs sampling with a bivariate normal, where the correlation parameter is not fixed or known. In other words, what is the conditional distribution of the ...
2
votes
1answer
66 views

Gibbs sampling for Multivariate: how to update?

In this page of Murphy's 'Machine Learning: a Probabilistic Perspective' it's explained how to do Gibbs sampling on a Gaussian Mixture Model. Reading this, I was trying to understand when to update ...
4
votes
2answers
140 views

R alternatives to JAGS/BUGS [closed]

I've recently fit more complex hidden markov models with random effects and covariates etc. JAGS was the only program that could get the job done. Now I want to write my own functions to facilitate ...
1
vote
1answer
68 views

Stationarity of coefficients when sampling VAR by Gibbs sampler

I am using Gibbs Sampler for VAR and have noticed that some researchers check the stationarity of $\beta$ coefficients while drawing. I am not sure why do they do that? Bayesian VARs do not require ...
0
votes
0answers
58 views

Derivation of a Gibbs sampler for a Bayesian model with hierarchical Dirichlet prior

I am studying Gibbs sampling. In particular, I got stuck on deriving Gibbs sampling when the reference Bayesian model has hierarchical Dirichlet distributions. As an example, let us start with the ...
2
votes
1answer
52 views

Gibbs Sampling - Calculating the full conditionals from the joint density

Given a joint density, $f(x_1, x_2)$, can its pmf/pdf be found generally by the method outlined below: For a joint density, $f(x_1, x_2)$ if we hold $x_2$ constant in the joint density, we will get ...
3
votes
1answer
186 views

Do I need to evaluate acceptance rates in Metropolis within Gibbs algorithm?

Consider the Gibbs sampler Sample $\theta' \sim p(\theta|\tau, D)$ Sample $\tau' \sim p(\tau|\theta', D)$ where $\theta,\tau$ parameters of the data $D$. Now assume that we can only sample from $p(\...
1
vote
0answers
37 views

Sampling states of an “unnatural” Hamiltonian System

I would like to sample from a Gibbs distribution given by $$f(p, q) = \frac{1}{\mathcal{Z}}e^{-H(p, q; \omega, J)}$$ where $H$ is the Hamiltonian on generalized coordinates $(p,q)\in \mathbb{R}^{2n}...
1
vote
0answers
22 views

Why the nodes in a Boltzmann machine need to be sampled one at a time?

Typically, we use Gibbs sampling to update (or generate samples from) energy based models. This means we update each node while keeping its markov blanket constant. Why can't we update/sample all ...
0
votes
1answer
273 views

What does MCMC do during burn-in period?

I am studying mcmc and I am wondering what mcmc does during burn-in period. And also what is the difference during burning period and after the burn-in period?
1
vote
1answer
165 views

What is the role of simulated annealing in Gibbs sampling?

While I was reading about Gibbs sampling, I happened to see "simulated annealing" but what is it doing in Gibbs sampling? Although I don't understand the full context of simulated annealing, I am ...
0
votes
0answers
27 views

Does MCMC Gibbs sampling algorithm first build a steady Markov Chain, then does the sampling to build the posterior distribution?

I am currently studying MCMC Gibbs sampling and while reading this part, a question has come into my head if MCMC Gibbs sampling first build a steady Markov Chain and does the sampling or does ...
2
votes
2answers
85 views

Need to understand a statement for Random Walk Metropolis algorithm's proposal distribution?

I was told that the proposal distribution of Random Walk Metropolis needs to be symmetric. But today I was reading a book about Bayesian Analysis which contains the following statement: "The proposal ...
1
vote
0answers
227 views

Bayesian Gamma Regression Update

I'm looking for a resource that explains how to do update the coefficients for a Bayesian gamma regression using Gibbs sampling. Specifically, if $y_i \sim Gamma(\alpha,\beta_i)$ and my data ...

1
2 3 4 5
8