Questions tagged [gibbs]

The Gibbs sampler is a simple form of Markov Chain Monte Carlo simulation, widely used in Bayesian statistics, based on sampling from full conditional distributions for each variable or group of variables. The name comes from the method being first used on Gibbs random fields modeling of images by Geman and Geman (1984).

Filter by
Sorted by
Tagged with
1
vote
1answer
35 views

Why iterations of Gibbs sampling for a bivariate Gaussian distribution can be seen as random walk?

In Section 4.4 of the excellent technical report Probabilistic Inference using Markov Chain Monte Carlo Methods, the author tries to analyze the performance of Gibbs and Metropolis algorithm with ...
1
vote
0answers
16 views

Plotting a random walk on R [closed]

I've run a Gibbs sampler and obtained a sample for $X_1$ and $X_2$. I'm trying to recreate a plot like this one: How do I recreate the walk part on R?
3
votes
1answer
74 views

Implementation of a blocked Gibbs sampler for a mixture model with a Dirichlet-process prior

I am trying to understand and implement the blocked Gibbs sampler described on page 552 in Bayesian Data Analysis by Gelman et al. in the context of using a Dirichlet process as a prior in a mixture ...
4
votes
1answer
135 views

Sampling from multivariate normal conditional on a negative minimum

Let $X\sim \mathcal{N}(\mu,\Sigma)$, where $\mu\in\mathbb{R}^n$ and $\Sigma\in\mathbb{R}^{n\times n}$. How can I efficiently sample from $X | {\min{X}\le 0}$? (I.e. from the distribution of $X$ ...
0
votes
0answers
8 views

How to sample posterior distribution for models with random effects?

I have a time series model contains some fixed parameters ($\beta_{1}$, k, m, etc. ) and also a random effect (i.e. $\beta_{t}$ follows a random walk, with starting value $\beta_{1}$ and variance ...
0
votes
0answers
19 views

How to improve mixing and effective size using Gibb's Sampling?

I have some data and I am trying to fit a Bayesian model. There are two things that I am unsure of: The mixing of my trace plots The effective size So I know that I can improve mixing by increasing ...
3
votes
1answer
139 views

Gibbs Sampling vs. Using Raw Probability in Contrastive Divergence

In Hinton's Practical Guide to Training Restricted Boltzmann Machines, Section 3, he discusses different situations in which one should take a sample from the Gibbs sampling process, and other ...
0
votes
1answer
69 views

How to built Gibbs sampler of Mixture Bayesian regression in R?

I am working on a Gibbs sampler of three parameters and we know the full conditional distribution of three parameters.
3
votes
1answer
194 views

Importance weight of conditioned particle in conditional SMC

In a generic particle filter, I understand the importance weights for each particle are calculated as $w_t^s \propto w_{t-1}^s \frac{p(y_t \mid z_t^s) p(z_t^s \mid z_{t-1}^s)}{q(z_t^s \mid z_{t-1}^s, ...
1
vote
0answers
20 views

Particle Gibbs Sampler For Regime-Switching Nonlinear Gaussian SSM

I'm reading this paper on using a non-linear Gaussian SSM for measuring regime-switching leverage effect using stock market data. I'm using it as jump-off point for an undergraduate paper. My advisor ...
2
votes
0answers
21 views

Gibbs updating algorithm (Gibbs steps) for computationally expensive likelihood

I am looking for a good way to update steps in a Gibbs sampler where the likelihood function is computationally expensive. Here is what I tried so far: By default JAGS uses a slice sampler. However, ...
0
votes
0answers
24 views

Deriving a full conditional distribution when Half-t distribution is used

I'm trying to use a Half-t distribution with Gibbs sampling to form a model. but having hard time finding the full conditional distribution. Suppose $$Y = X \beta + \epsilon $$, where $Y$ is a ...
2
votes
1answer
258 views

What does it means of Normalization term of Gibbs distribution?

I am studying about Gibbs distribution concept and I am confusing a one term in that concept that is normalization term. According to the Hammersley–Clifford theorem, an random $x$ can equivalently be ...
1
vote
0answers
12 views

Gibbs sampling Bayesian conditional distribution for mean of a Normal distribution

first post here in CV. I'm currently working on a textbook exercise on Gibbs Sampling and got stuck on naming the distribution for one of the conditional distributions. Question Consider a normal ...
2
votes
0answers
40 views

Gibbs sampler of a generative model

I understand what a Gibbs sampler is and I understand how LDA does classification. But I'm unsure how I can generate a Gibbs sampler for an LDA model and how to meld the two concepts. Let's say I ...
4
votes
1answer
72 views

Bayesian mixture model joint posterior

I am just starting to learn about bayesian mixture models. There is a few clarifications that I want to make which I am not sure myself. The graphical model below describes a gaussian mixture model ...
1
vote
2answers
131 views

Deriving full conditionals from joint distributions?

In this link (https://www.youtube.com/watch?v=a_08GKWHFWo), the author derives the conditional distributions from the joint; but I got lost in the mechanics of what happened, the process was overly ...
1
vote
0answers
36 views

Gibbs sampling proposals for bivariate normal?

I'm very familiar with Metropolis-Hastings, having implemented the algorithm myself to handle "toy problems." Gibbs sampling, however, is a bit trickier for me as I'm not quite certain what ...
2
votes
0answers
43 views

Intuition on why Gibbs Sampling samples from the posterior distribution

I am new to Gibbs Sampling and I do understand how the algorithm works but I would also like to understand how sampling from the conditional distributions is equivalent to sampling from the joint. ...
0
votes
0answers
12 views

Uncorrelated Samples from a non-conjugate (but well behaved) posterior

I'm trying to create a Dirichlet process mixture model with a kernel distribution similar to a product of gammas. (in fact, if I generate a latent random variable, it IS a product of (independent) ...
0
votes
1answer
166 views

How to create a distribution and sample?

Suppose we are given some small set of data on bundles of electrical wires and increasing voltages run through them, and we note how many of the individual wires fail. So for example, a large data ...
1
vote
0answers
18 views

How does pymc3 posterior simulation work in this simple case without having the full conditional distributions?

I'm trying to estimate the posterior distribution of the gamma parameters alpha and beta given that my data comes from a gamma distribution and the priors I chose come from two uniform distributions. ...
0
votes
1answer
33 views

Running several MCMC chains after convergence?

I am running a MCMC Gibbs sampler for a computationally expensive model. It takes ~12 hours to obtain 1000 iterations of this MCMC sampler. I have tested the sampler, and I found that the chain seems ...
0
votes
0answers
37 views

Estimating parameters with Gibbs sampling?

I've been trying to understand Gibbs sampling; my end goal is to intuitively understand it in the context of MCMC methods. However, in order to reach that end, I started a with simpler example. I ...
0
votes
0answers
14 views

Stationary distibution of Gibbs sampler

I am suppose to find stationary distribution of a Marcov chain generated by the following Gibbs sampler $n := 0$; $X_0 = x_0$; $(x_0>0)$ repeat Gen $Y_n \sim U(0, \exp(-X_n))$; Gen $X_{n+1} \...
0
votes
0answers
59 views

python gibbs sampler for bivariate normal distribution, failing to converge

I've been trying to understand Gibbs sampling for some time. Recently, I saw a video that made a good deal of sense. https://www.youtube.com/watch?v=a_08GKWHFWo The author used Gibbs sampling to ...
4
votes
2answers
138 views

What is the relationship between Boltzmann / Gibbs sampling and the softmax function?

I'm looking at sampling functions in the context of reinforcement learning; specifically the explore/exploit problem. A method I've seen pretty often is to derive the action by assigning a score to ...
0
votes
1answer
41 views

Why does my Gibbs sampler find two optimals?

[EDITED] I am using a Gibbs Sampler to find a Bayesian optimization to my multilevel (hierarchical) model (2 levels). However, when I run multiple chains (each chain having different starting values) ...
0
votes
1answer
41 views

Gibbs Sampler Conditional Marginal Computation

I have the following question regarding the Gibbs sampler, although it might be considered a simple question on conditional probability. For sake of simplicity, let us say we are trying to sample ...
1
vote
1answer
43 views

Relationship between variational inference and sampling in a Boltmzann-machine-like network

In this paper concerning a Boltzmann-machine-like network and its variational mean field approximation, the authors write In the stochastic system as well as the deterministic system, units evolve ...
0
votes
0answers
8 views

Inference on Author model

The Author Model is an LDA based model that first time introduced in paper [The Author-Topic Model for Authors and Documents]. I have studied the inference of the LDA model and know how to obtain the ...
1
vote
0answers
36 views

parameter estimation on the LDA model

I have a problem with estimating the parameters of $\theta$, and $\phi$ in the Latent Dirichlet Allocation (LDA) model. The article Finding scientific topics has done the estimation of the parameters ...
-1
votes
1answer
39 views

Which sampling methods (MCMC or otherwise) can be used if the posterior distribution is unknown?

The goal is to sample the posterior distribution of parameters describing some model (fairly low dimensional, generally no more than 10 parameters at the absolute most, usually around 5), but I don't ...
0
votes
0answers
41 views

What pitfalls should we avoid with Heidelberger-Welch convergence

I'm working through validating a Bayesian mixture model for multi-species occupancy with a collaborator. Initially, we relied on coda::heidel.diag to alert us to ...
1
vote
1answer
55 views

Multi-steps direct forecasting in AR(2) model through bayesian estimation of the model

I'm estimating an AR(2) model using Bayesian methods through Gibbs sampling and I want to perform 4 step ahead multi-steps direct forecasts. Inside the MCMC loop in each iteration I'm drawing the ...
0
votes
1answer
123 views

Replicating an experiment on GMRF (Gaussian Markov Random Field)

I am trying to understand an experiment from this paper, specifically Section 5.2. In the paper, they propose a new algorithm for computing the log-determinant of sparse matrices, and in section 5 ...
0
votes
0answers
30 views

How to calculate marginal transition matrix given the joint probability distribution for Gibbs sampling?

If I have two variables x$_{1}$ and x$_{2}$ which both take values in {1,2,3}, and I have the table representing their joint probabilities, how can I then determine the transition matrix for the ...
2
votes
1answer
167 views

What should be the burn in period for Metropolis-within-Gibbs?

I need to get samples from an unnormalized distribution $p(\theta, \tau | D)$. However, sampling directly from the joint distribution with Metropolis-Hastings is hard, as the sampler rarely finds ...
1
vote
0answers
19 views

Gibbs sampler with adaptive linear transformation

It is a well known fact that linear transformations can dramatically improve the performances of a Gibbs sampler when a ridge-like joint likelihood function occurs. Can I make an algorithm that ...
0
votes
0answers
36 views

Building an hierarchical model

I am trying to understand hierarchic Bayesian models. I am following the example http://www.openbugs.net/Examples/Pumps.html. The exercise I am trying to complete relates to the same problem, with the ...
0
votes
0answers
41 views

Sampling from hyperprior with Gibbs sampling

Let's say I have some set of data, $\vec{y}$, where each element is sampled as $$ y_n \sim Normal(\mu,1/\tau) $$ where $\tau$ is the precision, $1/\sigma^2$, and I want to use Gibbs sampling, choosing ...
1
vote
1answer
54 views

Gibbs sampling example of a bivariate normal with unknown correlation

I'm looking for an example of using Gibbs sampling with a bivariate normal, where the correlation parameter is not fixed or known. In other words, what is the conditional distribution of the ...
2
votes
1answer
66 views

Gibbs sampling for Multivariate: how to update?

In this page of Murphy's 'Machine Learning: a Probabilistic Perspective' it's explained how to do Gibbs sampling on a Gaussian Mixture Model. Reading this, I was trying to understand when to update ...
4
votes
2answers
139 views

R alternatives to JAGS/BUGS [closed]

I've recently fit more complex hidden markov models with random effects and covariates etc. JAGS was the only program that could get the job done. Now I want to write my own functions to facilitate ...
1
vote
1answer
68 views

Stationarity of coefficients when sampling VAR by Gibbs sampler

I am using Gibbs Sampler for VAR and have noticed that some researchers check the stationarity of $\beta$ coefficients while drawing. I am not sure why do they do that? Bayesian VARs do not require ...
0
votes
0answers
58 views

Derivation of a Gibbs sampler for a Bayesian model with hierarchical Dirichlet prior

I am studying Gibbs sampling. In particular, I got stuck on deriving Gibbs sampling when the reference Bayesian model has hierarchical Dirichlet distributions. As an example, let us start with the ...
2
votes
1answer
52 views

Gibbs Sampling - Calculating the full conditionals from the joint density

Given a joint density, $f(x_1, x_2)$, can its pmf/pdf be found generally by the method outlined below: For a joint density, $f(x_1, x_2)$ if we hold $x_2$ constant in the joint density, we will get ...
3
votes
1answer
185 views

Do I need to evaluate acceptance rates in Metropolis within Gibbs algorithm?

Consider the Gibbs sampler Sample $\theta' \sim p(\theta|\tau, D)$ Sample $\tau' \sim p(\tau|\theta', D)$ where $\theta,\tau$ parameters of the data $D$. Now assume that we can only sample from $p(\...
1
vote
0answers
37 views

Sampling states of an “unnatural” Hamiltonian System

I would like to sample from a Gibbs distribution given by $$f(p, q) = \frac{1}{\mathcal{Z}}e^{-H(p, q; \omega, J)}$$ where $H$ is the Hamiltonian on generalized coordinates $(p,q)\in \mathbb{R}^{2n}...
2
votes
2answers
239 views

How does this Sampler work for the Concentration parameter of Dirichlet Process?

I am puzzled by how this Gibbs sampler on section 6 of Escobar & West (1995) works. To put it in simple words, the aim is to sample $\alpha$. The defined terms are: $$\eta\sim \texttt{Beta}(a,b)$$ ...

1
2 3 4 5
8