Questions tagged [gibbs]

The Gibbs sampler is a simple form of Markov Chain Monte Carlo simulation, widely used in Bayesian statistics, based on sampling from full conditional distributions for each variable or group of variables. The name comes from the method being first used on Gibbs random fields modeling of images by Geman and Geman (1984).

Filter by
Sorted by
Tagged with
0
votes
0answers
16 views

Incorrect inference of conditional density in simple linear model and Gibbs sampling

I am studying Bayesian inference and just found out that I have a major misunderstanding of the core concepts behind theory of probability and linear models. Assume the following model: $$y = \epsilon$...
1
vote
1answer
81 views

Applying outlier adjustment using student's t distribution in a state-space model

I'm exploring performing outlier adjustment in a state-space model by using student's $t$ distribution. The gist of the problem is formulated as follows: $$ \begin{align*} y_t^* &= u_t + o_t - o_{...
2
votes
1answer
220 views

Gibbs Sampler for Normal and Inverse Gamma Distribution in R

I'm trying to implement a Gibbs sampler for the following conditional distributions using R: This is the code I have in R so far: ...
0
votes
0answers
16 views

Posterior conditionals for bivariate normal

I'm new to bayesian statistics and I'm studying Bayesian models. I'm having trouble writing the Gibbs sampler for a particular case of bivariate normals. Assume now that $d_i = (x_i, y_i)$ for $i=1,2\...
0
votes
0answers
35 views

Covariance matrix of multivariate Gaussian distribution

I have a Gibbs Sampling problem in which to sample the initial values of [x] variables where x=[x1 x2 x3 x4] represented by Multi variate normal distribution: It is given that each element of [x] has ...
1
vote
0answers
29 views

MCMC algorithm for Hierarchical Bayes model with variable number of mixture components

I am trying to develop an MCMC algorithm for clustering $n$ data-points $y_{1},y_{2},\dots,y_{n}$ using a Gaussian mixture model, but with a prior defined on the number of components K. The ...
0
votes
0answers
13 views

How to estimate variance in Bayesian matrix factorization using Gibbs samples?

I have implemented a Gibbs sampler for Bayesian Matrix Factorization /Completion of matrix $R = (r_{ij})$ which is $(N, M)$ dimensional and $p(r_{ij} | \textbf{u}_i, \textbf{v}_j) = N(r_{ij}|\textbf{u}...
0
votes
0answers
12 views

Sampling Complexity for Gibbs sampling

I am trying to understand what is the sampling complexity of the best-known classical algorithm (along the lines of contrastive divergence etc) for drawing a sample from Gibbs distribution in terms of ...
0
votes
0answers
13 views

How to find conditional density?

Suppose there is a unit $\mathcal{l}_p$ ball $$ B_p = \{ x \vert \sum^n_{i=1}\vert x_i \vert^p \le 1\}, $$ where $p \ge 1$, $n$ is the dimension of the space and $x_i$ is the $i$-th coordinate of $x$. ...
1
vote
0answers
15 views

Gibbs sampling step for variables that have a complex offline prior in an MCMC hybrid

I have a question about how to use an offline function as a prior when performing a Gibbs/hybrid analysis. Let's say I have data $y$ and some parameters which I'll simplify to $\theta_1, \theta_2$. ...
0
votes
1answer
38 views

Expanding conditional probability for Gibbs sampling with many parameters

I'm trying to use Gibbs sampling to get the following target distribution: $$ p(a,b,c \lvert x, z) $$ Where $z = f(x,a,b,c)$ and the rest are independent. I know the following conditional ...
1
vote
1answer
39 views

Why iterations of Gibbs sampling for a bivariate Gaussian distribution can be seen as random walk?

In Section 4.4 of the excellent technical report Probabilistic Inference using Markov Chain Monte Carlo Methods, the author tries to analyze the performance of Gibbs and Metropolis algorithm with ...
1
vote
0answers
17 views

Plotting a random walk on R [closed]

I've run a Gibbs sampler and obtained a sample for $X_1$ and $X_2$. I'm trying to recreate a plot like this one: How do I recreate the walk part on R?
0
votes
0answers
12 views

How to sample posterior distribution for models with random effects?

I have a time series model contains some fixed parameters ($\beta_{1}$, k, m, etc. ) and also a random effect (i.e. $\beta_{t}$ follows a random walk, with starting value $\beta_{1}$ and variance ...
0
votes
1answer
127 views

How to built Gibbs sampler of Mixture Bayesian regression in R?

I am working on a Gibbs sampler of three parameters and we know the full conditional distribution of three parameters.
1
vote
0answers
24 views

Particle Gibbs Sampler For Regime-Switching Nonlinear Gaussian SSM

I'm reading this paper on using a non-linear Gaussian SSM for measuring regime-switching leverage effect using stock market data. I'm using it as jump-off point for an undergraduate paper. My advisor ...
2
votes
0answers
28 views

Gibbs updating algorithm (Gibbs steps) for computationally expensive likelihood

I am looking for a good way to update steps in a Gibbs sampler where the likelihood function is computationally expensive. Here is what I tried so far: By default JAGS uses a slice sampler. However, ...
0
votes
0answers
29 views

Deriving a full conditional distribution when Half-t distribution is used

I'm trying to use a Half-t distribution with Gibbs sampling to form a model. but having hard time finding the full conditional distribution. Suppose $$Y = X \beta + \epsilon $$, where $Y$ is a ...
1
vote
0answers
18 views

Gibbs sampling Bayesian conditional distribution for mean of a Normal distribution

first post here in CV. I'm currently working on a textbook exercise on Gibbs Sampling and got stuck on naming the distribution for one of the conditional distributions. Question Consider a normal ...
2
votes
0answers
45 views

Gibbs sampler of a generative model

I understand what a Gibbs sampler is and I understand how LDA does classification. But I'm unsure how I can generate a Gibbs sampler for an LDA model and how to meld the two concepts. Let's say I ...
4
votes
1answer
88 views

Bayesian mixture model joint posterior

I am just starting to learn about bayesian mixture models. There is a few clarifications that I want to make which I am not sure myself. The graphical model below describes a gaussian mixture model ...
4
votes
1answer
110 views

Implementation of a blocked Gibbs sampler for a mixture model with a Dirichlet-process prior

I am trying to understand and implement the blocked Gibbs sampler described on page 552 in Bayesian Data Analysis by Gelman et al. in the context of using a Dirichlet process as a prior in a mixture ...
1
vote
2answers
289 views

Deriving full conditionals from joint distributions?

In this link (https://www.youtube.com/watch?v=a_08GKWHFWo), the author derives the conditional distributions from the joint; but I got lost in the mechanics of what happened, the process was overly ...
1
vote
0answers
130 views

Gibbs sampling proposals for bivariate normal?

I'm very familiar with Metropolis-Hastings, having implemented the algorithm myself to handle "toy problems." Gibbs sampling, however, is a bit trickier for me as I'm not quite certain what ...
2
votes
0answers
61 views

Intuition on why Gibbs Sampling samples from the posterior distribution

I am new to Gibbs Sampling and I do understand how the algorithm works but I would also like to understand how sampling from the conditional distributions is equivalent to sampling from the joint. ...
0
votes
0answers
13 views

Uncorrelated Samples from a non-conjugate (but well behaved) posterior

I'm trying to create a Dirichlet process mixture model with a kernel distribution similar to a product of gammas. (in fact, if I generate a latent random variable, it IS a product of (independent) ...
1
vote
0answers
18 views

How does pymc3 posterior simulation work in this simple case without having the full conditional distributions?

I'm trying to estimate the posterior distribution of the gamma parameters alpha and beta given that my data comes from a gamma distribution and the priors I chose come from two uniform distributions. ...
0
votes
1answer
39 views

Running several MCMC chains after convergence?

I am running a MCMC Gibbs sampler for a computationally expensive model. It takes ~12 hours to obtain 1000 iterations of this MCMC sampler. I have tested the sampler, and I found that the chain seems ...
0
votes
0answers
41 views

Estimating parameters with Gibbs sampling?

I've been trying to understand Gibbs sampling; my end goal is to intuitively understand it in the context of MCMC methods. However, in order to reach that end, I started a with simpler example. I ...
0
votes
0answers
14 views

Stationary distibution of Gibbs sampler

I am suppose to find stationary distribution of a Marcov chain generated by the following Gibbs sampler $n := 0$; $X_0 = x_0$; $(x_0>0)$ repeat Gen $Y_n \sim U(0, \exp(-X_n))$; Gen $X_{n+1} \...
0
votes
0answers
121 views

python gibbs sampler for bivariate normal distribution, failing to converge

I've been trying to understand Gibbs sampling for some time. Recently, I saw a video that made a good deal of sense. https://www.youtube.com/watch?v=a_08GKWHFWo The author used Gibbs sampling to ...
4
votes
2answers
248 views

What is the relationship between Boltzmann / Gibbs sampling and the softmax function?

I'm looking at sampling functions in the context of reinforcement learning; specifically the explore/exploit problem. A method I've seen pretty often is to derive the action by assigning a score to ...
0
votes
1answer
46 views

Why does my Gibbs sampler find two optimals?

[EDITED] I am using a Gibbs Sampler to find a Bayesian optimization to my multilevel (hierarchical) model (2 levels). However, when I run multiple chains (each chain having different starting values) ...
0
votes
1answer
65 views

Gibbs Sampler Conditional Marginal Computation

I have the following question regarding the Gibbs sampler, although it might be considered a simple question on conditional probability. For sake of simplicity, let us say we are trying to sample ...
1
vote
0answers
54 views

parameter estimation on the LDA model

I have a problem with estimating the parameters of $\theta$, and $\phi$ in the Latent Dirichlet Allocation (LDA) model. The article Finding scientific topics has done the estimation of the parameters ...
1
vote
1answer
52 views

Relationship between variational inference and sampling in a Boltmzann-machine-like network

In this paper concerning a Boltzmann-machine-like network and its variational mean field approximation, the authors write In the stochastic system as well as the deterministic system, units evolve ...
0
votes
0answers
51 views

What pitfalls should we avoid with Heidelberger-Welch convergence

I'm working through validating a Bayesian mixture model for multi-species occupancy with a collaborator. Initially, we relied on coda::heidel.diag to alert us to ...
1
vote
1answer
87 views

Multi-steps direct forecasting in AR(2) model through bayesian estimation of the model

I'm estimating an AR(2) model using Bayesian methods through Gibbs sampling and I want to perform 4 step ahead multi-steps direct forecasts. Inside the MCMC loop in each iteration I'm drawing the ...
-1
votes
1answer
56 views

Which sampling methods (MCMC or otherwise) can be used if the posterior distribution is unknown?

The goal is to sample the posterior distribution of parameters describing some model (fairly low dimensional, generally no more than 10 parameters at the absolute most, usually around 5), but I don't ...
0
votes
1answer
171 views

Replicating an experiment on GMRF (Gaussian Markov Random Field)

I am trying to understand an experiment from this paper, specifically Section 5.2. In the paper, they propose a new algorithm for computing the log-determinant of sparse matrices, and in section 5 ...
2
votes
1answer
242 views

What should be the burn in period for Metropolis-within-Gibbs?

I need to get samples from an unnormalized distribution $p(\theta, \tau | D)$. However, sampling directly from the joint distribution with Metropolis-Hastings is hard, as the sampler rarely finds ...
1
vote
0answers
19 views

Gibbs sampler with adaptive linear transformation

It is a well known fact that linear transformations can dramatically improve the performances of a Gibbs sampler when a ridge-like joint likelihood function occurs. Can I make an algorithm that ...
1
vote
1answer
68 views

Gibbs sampling example of a bivariate normal with unknown correlation

I'm looking for an example of using Gibbs sampling with a bivariate normal, where the correlation parameter is not fixed or known. In other words, what is the conditional distribution of the ...
2
votes
1answer
89 views

Gibbs sampling for Multivariate: how to update?

In this page of Murphy's 'Machine Learning: a Probabilistic Perspective' it's explained how to do Gibbs sampling on a Gaussian Mixture Model. Reading this, I was trying to understand when to update ...
4
votes
2answers
192 views

R alternatives to JAGS/BUGS [closed]

I've recently fit more complex hidden markov models with random effects and covariates etc. JAGS was the only program that could get the job done. Now I want to write my own functions to facilitate ...
1
vote
1answer
70 views

Stationarity of coefficients when sampling VAR by Gibbs sampler

I am using Gibbs Sampler for VAR and have noticed that some researchers check the stationarity of $\beta$ coefficients while drawing. I am not sure why do they do that? Bayesian VARs do not require ...
2
votes
1answer
54 views

Gibbs Sampling - Calculating the full conditionals from the joint density

Given a joint density, $f(x_1, x_2)$, can its pmf/pdf be found generally by the method outlined below: For a joint density, $f(x_1, x_2)$ if we hold $x_2$ constant in the joint density, we will get ...
3
votes
1answer
260 views

Do I need to evaluate acceptance rates in Metropolis within Gibbs algorithm?

Consider the Gibbs sampler Sample $\theta' \sim p(\theta|\tau, D)$ Sample $\tau' \sim p(\tau|\theta', D)$ where $\theta,\tau$ parameters of the data $D$. Now assume that we can only sample from $p(\...
1
vote
0answers
39 views

Sampling states of an “unnatural” Hamiltonian System

I would like to sample from a Gibbs distribution given by $$f(p, q) = \frac{1}{\mathcal{Z}}e^{-H(p, q; \omega, J)}$$ where $H$ is the Hamiltonian on generalized coordinates $(p,q)\in \mathbb{R}^{2n}...
1
vote
0answers
22 views

Why the nodes in a Boltzmann machine need to be sampled one at a time?

Typically, we use Gibbs sampling to update (or generate samples from) energy based models. This means we update each node while keeping its markov blanket constant. Why can't we update/sample all ...

1
2 3 4 5
8