Questions tagged [gibbs]

The Gibbs sampler is a simple form of Markov Chain Monte Carlo simulation, widely used in Bayesian statistics, based on sampling from full conditional distributions for each variable or group of variables. The name comes from the method being first used on Gibbs random fields modeling of images by Geman and Geman (1984).

Filter by
Sorted by
Tagged with
3
votes
2answers
2k views

Sampling variables and calculating likelihood in WinBUGS/OpenBUGS

I am trying to read some WinBUGS/OpenBUGS examples to figure out how to specify models. I can't seem to understand where the probabilistic dnorm, ...
3
votes
1answer
3k views

Gibbs sampling convergence

In an astronomical context, the authors of a paper desire to use a Gibbs algorithm. Please note: I am inexperience in MCMC algorithms, and specifically in Gibbs sampling. What we want, in essence, is ...
3
votes
1answer
244 views

Gibbs sampling from conditional full posterior distribution

I am reading the paper by Willemsen et al (2015), "A multivariate Bayesian model for embryonic growth", Statistics in Medicine, 34:8, 1351–1365 where they define the posterior distribution as, \begin{...
3
votes
1answer
61 views

Bayesian mixture model joint posterior

I am just starting to learn about bayesian mixture models. There is a few clarifications that I want to make which I am not sure myself. The graphical model below describes a gaussian mixture model ...
3
votes
2answers
106 views

R alternatives to JAGS/BUGS [closed]

I've recently fit more complex hidden markov models with random effects and covariates etc. JAGS was the only program that could get the job done. Now I want to write my own functions to facilitate ...
3
votes
1answer
870 views

Gibbs sampling and Conjugate Priors

Are conjugate priors required when performing Gibbs sampling?
3
votes
1answer
5k views

JAGS: posterior predictive distribution

I am fitting a simple linear regression model with R & JAGS: ...
3
votes
1answer
158 views

Do I need to evaluate acceptance rates in Metropolis within Gibbs algorithm?

Consider the Gibbs sampler Sample $\theta' \sim p(\theta|\tau, D)$ Sample $\tau' \sim p(\tau|\theta', D)$ where $\theta,\tau$ parameters of the data $D$. Now assume that we can only sample from $p(\...
3
votes
1answer
138 views

What's the role of the data in Gibbs Sampling?

Trying to wrap my mind around Gibbs Sampling. Across many answers in this same forum, I constantly notice that the examples shown do not actually require an observed data set (First example (with R ...
3
votes
1answer
218 views

What is a hierarchical model that can estimated via the Metropolis-Hastings Algorithm but not the Gibbs Sampler?

My understanding of the differences between MH and Gibbs Samplers is that a Gibbs Sampler is usually used when the full conditionals are present to us. In other words, it is a known distribution, so ...
3
votes
2answers
489 views

Gibbs sampling and mixed distribution

For a project, I need to simulate from a joint distribution with both continuous and discrete variables that are dependent. The conditional distribution of any variable given the rest is known. I ...
3
votes
2answers
843 views

Markov Chain Monte Carlo (MCMC) with transformed data

I want to obtain an estimate of a parameter $\Theta$ in a model for a random variable $X$ dependent on $\Theta$ with known but complicated likelihood $L(\Theta|X) = p(X|\Theta)$. $X$ is not directly ...
3
votes
1answer
1k views

Gibbs sampling with mixed prior using a Metropolis-Hastings step

My questions are about a sampling procedure for fitting a Bayesian hierarchical model where one of the priors is a mixture distribution of discrete and continuous parts. The model is not my own but I ...
3
votes
1answer
33 views

Slice Sampling asks to draw from $f^{-1}]y,+\infty[$

Slice Sampling asks to draw uniformly from $f^{-1}]y,+\infty[$. Wikipedia page However, how can we be sure that a uniform defined over the set $f^{-1}]y,+\infty[$ is in fact proper? If I had to ...
3
votes
1answer
223 views

Derive the Gibbs sampler for this bivariate distribution

I understand the theory of Gibbs sampling. It is an iterative sampling algorithm that defines, sequence of random variables with the property of a Markov chain. Specifically, I choose any starting ...
3
votes
1answer
1k views

Gibbs measure and normal distribution

On Wikipedia, the Gibbs measure defines the probability as: $$ P(X=x) = \frac{1}{Z(\beta)}\exp(-\beta E(x)) $$ Now, the familiar form of the normal distribution is: $$ P(x) = \frac{1}{\sqrt{2\pi}\...
3
votes
1answer
923 views

Prior selection for Gaussian Processes (GP)

I am trying to select a prior for the covariance parameters of my Gaussian Process (GP) and have been running into numerical problems with my MCMC code. My model is the following: $$Y = D\beta + GP(...
3
votes
1answer
2k views

Sampling from conditional distribution in general case

I'm dealing with Gibbs Sampling now. Let's consider the example: I know the distribution of X|Y and the distribution of Y. They are some known - Binomial or Beta or other but particular. Thus I have ...
3
votes
1answer
521 views

Prior of multivariate Polya distribution?

Anyone knows a prior (preferably conjugate) to the multivariate Polya distribution? I need it for Gibbs sampling. So if anyone has another idea, I am interested.
3
votes
1answer
82 views

Auxiliary variable Gibbs sampler

Suppose we want to sample from a pdf $f(x_1,x_2)$. It's easy to sample from $x_1 \vert x_2$, but not $x_2 \vert x_1$, so we introduce an auxiliary variable $u$ such that $\int f(x_1,x_2,u) du = f(x_1,...
3
votes
1answer
190 views

Importance weight of conditioned particle in conditional SMC

In a generic particle filter, I understand the importance weights for each particle are calculated as $w_t^s \propto w_{t-1}^s \frac{p(y_t \mid z_t^s) p(z_t^s \mid z_{t-1}^s)}{q(z_t^s \mid z_{t-1}^s, ...
3
votes
1answer
134 views

Sampling posterior of empty cluster in GMM and Gibbs

Consider performing inference via a standard Gibbs sampler for a standard Gaussian Mixture Model (GMM) with $k$ components that are Gaussians $$\mathcal{N}(\mu_{k}, \sigma^{2}_{k})$$ where we assume ...
3
votes
1answer
589 views

Blocked Gibbs Sampling using Forward / Backward Algorithm

I am new to machine learning and have been reading about gibbs sampling. From my understanding, a Gibbs algorithm samples a single variable iteratively conditioned on all other variables. In blocked ...
3
votes
1answer
360 views

Gibbs Sampler - Sample mean convergence

To simulate from the posterior distribution $p(\theta|Y)$ where $\theta = (\mu,\lambda_1,\lambda_2)$, I run a Gibbs sampler to draw approximately random values from $p(\theta|Y)$. This Gibbs sampler ...
3
votes
1answer
447 views

Need help deriving a gibbs sampler for a normal mixture model with two components

Let $\theta_i$ be an indicator that the i-th eruption is a long eruption. (i.e. $\theta_i = 1$ if the i-th eruption is long and $\theta_i = 0$ otherwise.) Assume the following model and derive a Gibbs ...
3
votes
1answer
106 views

Posterior parameter distribution

I am considering the following non-linear state space model: $X_t=\frac{X_{t-1}}{2}+25\frac{X_{t-1}}{1+X_{t-1}^2}+8\cos{1.2t}+\epsilon_t, \epsilon_t\sim N(0,\sigma_x^2 ) $ $Y_t=\frac{X_t^2}{20}+\...
3
votes
1answer
172 views

Calculating conditional probability

Let's consider I have the pair of distributions: \begin{align} X|t &\sim Binomial(n,t) \\ t &\sim Beta(a,b)$ \end{align} Here $n,a,b$ are known. I need to construct conditional probability ...
3
votes
0answers
56 views

Implementation of a blocked Gibbs sampler for a mixture model with a Dirichlet-process prior

I am trying to understand and implement the blocked Gibbs sampler described on page 552 in Bayesian Data Analysis by Gelman et al. in the context of using a Dirichlet process as a prior in a mixture ...
3
votes
0answers
88 views

How to implement a M-H step in a Gibbs sampling

I am having trouble implementing a Metropolis Hastings step in a Gibbs sampling problem. The following code was taken from https://www.stat.colostate.edu/computationalstatistics/ Details: It is a ...
3
votes
1answer
132 views

Gibbs Sampling vs. Using Raw Probability in Contrastive Divergence

In Hinton's Practical Guide to Training Restricted Boltzmann Machines, Section 3, he discusses different situations in which one should take a sample from the Gibbs sampling process, and other ...
3
votes
0answers
849 views

Bayesian estimation of Dynamic Regression with AR(1) parameters

I would like to draw (Bayesian) inference in a dynamic linear regression with regression parameters following independent AR(1) processes $\beta_{t,i} = \mu_i+\beta_{t-1,i}+w_{t,i}$. However, I ...
3
votes
0answers
370 views

When Gibbs Sampling is fast/slow to converge?

Are there any heuristic/theories showing that on what kind of Bayesian models the convergence of Gibbs sampling is fast/slow? For example, from my limited experience, I feel (may be wrong) when a ...
3
votes
0answers
744 views

Criteria in determining “step size” of Metropolis-hasting algorithms

I am training a complex Bayesian model using Gibbs sampling and Metropolis-Hasting algorithm. Most of the parameters are directly sampled by using conjugate priors except for 3 params which are ...
3
votes
0answers
138 views

Seeking help in Bayesian Mixed Effects Model

I am implementing a Bayesian Mixed Effects model in my research problem. The model is written as, $y_i = X_i(\alpha + \beta_i) + \epsilon_i$, where $i = 1, 2, \ldots, m$ is the index of response, $j = ...
3
votes
0answers
210 views

When 2% of the Bayesian Model have not converged?

I have model with 20000 latent parameters, set up in a Gibb's sampler. 98% of the parameters and sometimes 99.5% of the parameters satisfy the Geweke convergence statistic, have low autocorrelation ...
3
votes
0answers
188 views

Derive Marginal Posterior to set up Gibbs-Sampler

I am currently trying to replicate a Hierarchical Model for multivariate returns proposed in the paper Portfolio selection using hierarchical Bayesian analysis and MCMC methods. However, in order to ...
3
votes
0answers
775 views

Efficiency in Metropolis Vs Gibbs sampling

I have read that Gibbs sampling is more efficient than Metropolis algorithm. Why? Is this due only to the fact the in Gibbs sampling the acceptance rate is $1$, so that the chain needs fewer ...
3
votes
0answers
2k views

How to understand Gibbs distribution

I have a graph model such as Following the Hammersley–Clifford theorem describes that Markov random fields exhibit a Gibbs distribution with an energy function as follows: $$P(x)=\frac {exp(-U(x))}{...
3
votes
1answer
145 views

Bayesian Mixture Model Gibbs Sampler for two linear relationships

I am attempting to use a Gibbs Sampler to model a mixture of two groups, where the group membership is defined by a linear relationship conditional on x. Both groups have the same slope and intercept, ...
3
votes
0answers
155 views

Gibbs sampler for local linear trend model

Question: Consider the local linear trend model given by: \begin{align*} y_t = \mu_t + \tau \varepsilon_t \ \cdots \ \text{Observation equation} \\ \mu_{t+1} = \phi \mu_t + \eta_t \ \cdots \ \text{...
3
votes
0answers
284 views

Sampling Stationary Vector Autoregression coefficients while Gibbs Sampling

I have been estimating a Bayesian Vector Autoregression using Gibbs Sampling. When constructing the posterior predictive distribution, I have noticed that when the simulated coefficients from the MCMC ...
2
votes
2answers
479 views

Gibbs sampler implementation

I am just getting started with the Gibbs Sampler and came across an implementation from here and here and here. All of theses implementations are based on the first article. There is an inner loop ...
2
votes
1answer
469 views

Gibbs sampling on the use of gamma distribution

I am trying to reproduce a Gibbs sampling but some points are unclear to me. Let's assume a linear model of the following form $$y=x\beta + \epsilon$$ The prior distribution for the beta ...
2
votes
2answers
2k views

How does Gibbs sampling produce values for a variable using the univariate conditional probability?

I have a question about Gibbs sampling for generating samples. The Gibbs sampling algorithm is often stated. $x^0 = (x_1^0, x_2^0, \ldots, x_n^0)$ //initialize random values for $t=1$ in $T$ //...
2
votes
1answer
693 views

Conditional distribution for Gibbs sampling for Gaussian mixture

If we draw $n$ i.i.d. points $x_1,x_2,\dots,x_n$ from the following Gaussian mixture: $$ \frac 12 \mathcal N(x \mid \mu_1,1) + \frac 12 \mathcal N(x\mid \mu_2,1) $$ and the prior $p(\mu_1 , \mu_2 )$ ...
2
votes
1answer
270 views

Why can Gibbs sampling outputs be used in Rao-Blackwellization?

I'm currently learning Chib (1995)'s method to calculate the marginal likelihood of a Bayesian model using Gibbs sampling outputs. I'm stuck in the Rao-Blackwellization step. Suppose $\mu$ and $\phi$...
2
votes
1answer
206 views

Is Coordinate Ascent algorithm related to Gibbs Sampling in some way?

I wonder if only me feel there are certain connections between them, I googled it for a long time, but found no where mentioned these two method. But to me, they indeed looked so related, Could anyone ...
2
votes
1answer
1k views

Implementing Gibbs sampler in R from posterior distribution

I am referencing a follow-up idea from something I posted earlier (Zero-inflated Poisson and Gibbs sampling, proofs and sampling). I want to implement the Gibbs sampler, by generating a large (...
2
votes
1answer
2k views

Gibbs sampling from posterior distribution using R

New to MCMC. I have a model, saying $$Y_i=\beta_0+\beta_1x_{i1}+\beta_2x_{i2}+\frac{e_i}{\sqrt{\mu}}$$ where $x_{ij}$ are fixed covariates, $e_i\sim N(0,1)$, $\beta_0$, $\beta_1$, $\beta_2$ and $\mu$ ...
2
votes
1answer
238 views

Drawing from a conditional density

I have a simple question. Suppose $X=(X_1,X_2,X_3)$ is multivariate normal. What's the best (quickest) way to draw from the conditional density $X_1\mid \exp(X_1)+\exp(X_2)+X_3$?

1 2
3
4 5
8