Questions tagged [gibbs]

The Gibbs sampler is a simple form of Markov Chain Monte Carlo simulation, widely used in Bayesian statistics, based on sampling from full conditional distributions for each variable or group of variables. The name comes from the method being first used on Gibbs random fields modeling of images by Geman and Geman (1984).

Filter by
Sorted by
Tagged with
0
votes
0answers
19 views

Gibbs sampling for simple posterior distribution?

I have a likelihood function, $$ p(x) = \theta^{\sum x} (1- \theta)^{n-\sum x} $$ and prior distribution, $$ p(\theta) \propto \theta^{\alpha - 1} (1- \theta)^{\beta - 1}$$ then the posterior ...
0
votes
0answers
7 views

Belief Propagation vs Gibbs Sampling

In general what are the cons and pros of using Gibbs sampling to estimate a complex posterior (assuming we can sample from the conditionals) over belief propagation (using a factor graph)?
1
vote
0answers
15 views

Markov chain Monte Carlo, Mixing Time

How do you estimate the mixing time for a markov chain? I read somewhere one can use the sum of the auto-correlation coefficients or the sum of the auto-covariance coefficients, but I cannot seem to ...
0
votes
0answers
14 views

sampling question in Gibbs sampling for a Gaussian mixture model

I have some confusions regarding the Gibbs sampling step for the following mixture model: consider a mixture model of the following generative process: $\theta \sim Dir(\alpha) $ (global hidden ...
1
vote
0answers
42 views

What's the point of Gibbs Sampling? [duplicate]

I am reading a book on doing Bayesian Data Analysis. I have just learned what the Metropolis Hastings (MH) Algo does, at least in relation to Bayesian Data Analysis. My understanding of the MH Algo ...
0
votes
0answers
18 views

the convergence speed for a Markov chain

For a metropolis hastings algorithm, suppose that the stationary distribution is defined as the Gibbs Boltzmann distribution $\pi_T(x)= \frac{1}{Z_T}e^{-\frac{V(x)}{T} }$ where $Z_T = \sum_{y\in V} e^{...
0
votes
0answers
17 views

Dirichlet Process mixture model with independent features

I'm trying to construct a Dirichlet process mixture model for clustering where the samples have independent features. In other words, to evaluate the likelihood of sample $x_i$, I would compute $\...
0
votes
1answer
21 views

Gibb's sampling where target prob distribution is itself a conditional joint distribution - p(x,y|t)

I'm new to Gibb's sampling and need basic guidance. Say p1,p2,q-> are Gaussian variables. p1->q<-p2 and q->x where x is a discrete variable (either 1 or 0). How do I go about Sampling (using ...
1
vote
1answer
23 views

Assign an error to the parameters of MAP estimate

Through a MCMC Gibbs sampler I obtain $M$ chains of the parameters vector $\mathbf{\theta}$, meaning that each component of $\mathbf{\theta}$ is the value of one parameter at a given iteration. ...
0
votes
1answer
123 views

GIBBS sampling : do samples for the subset of variables approxiamate the related marginal distribution?

I'm reading the page Gibbs sampling on wikipedia. I really don't understand why the following statement is true. "The marginal distribution of any subset of variables can be approximated by simply ...
1
vote
0answers
23 views

Is there something wrong with my Bayesian hierarchical spatio-temporal model?

I built a Bayesian spatio-temporal model and one of the parameters to be estimated is the random spatial effects s. The random spatial effect is assigned an intrinsic conditional autoregressive prior (...
4
votes
1answer
82 views

Is the joint distribution $P_{XY}(x,y)$ determined from the conditionals $P_{X|Y}(x|y)$ and $P_{Y|X}(y|x)$?

For simplicity assume that $X,Y$ are discrete, finite, random variables, with joint distribution $P_{XY}(x,y) = \mathbb{P}(X=x\wedge Y=y)$. Now suppose that we do not know $P_{XY}(x,y)$, but are ...
0
votes
0answers
32 views

sampling from joint distribution to recover marginal distribution

I'm going through Bayesian Core and have gotten stuck at this remark on page 233: " A first remark that motivates the use of the Gibbs sampler is that, within structures such as $$ \pi(x_1) = \int \...
0
votes
1answer
35 views

Gibbs sampling: Conjugate prior for a two component known-unknown variance?

If I have a normally distributed variable $\mathcal{N}(\mu,\frac{1}{\tau})$ with fixed $\mu$ then the conjugate prior for an unknown $\tau$ is then $\mathcal{Ga}(\frac{n}{2}+\alpha, \beta + \sum_i \...
0
votes
0answers
17 views

Any relationship between MAP full conditionals and MAP joint?

Context: Bayesian model that one can draw posterior samples from via Gibbs sampling of the relevant full conditionals. Question: Can anything be said (i.e. bounds, conditions for equality) about the ...
0
votes
0answers
12 views

Gibbs Sampling attempt at a simple Coxian distribution

I have the following Coxian model for inter-arival times ($x_i$) that has $C_x^2 < 1$: $$ p(x_i\mid \lambda,\theta) = \theta \lambda^r x_i^r e^{-\lambda x_i} + (1-\theta)\lambda e^{-\lambda x_i} $...
1
vote
0answers
37 views

Understand the Holmes and Held (2006) Bayesian probit MCMC algorithm

Holmes and Held (2006) suggest a simple approach to reduce autocorrelation in the MCMC algorithm proposed by Albert and Chib (1993). HH (2006) propose to update $\beta$ and $z$ jointly, making use of ...
0
votes
0answers
50 views

Gibbs Sampler for GMM

In Rasmussen's paper it is introduced a Gibbs sampler to make inference about a standard Gaussian Mixture Model. To simplify, assume the 1-d case with basic hierarchical structure, that is: $x_i|...
2
votes
0answers
27 views

Gibbs sampling for mixture with Dirichlet prior?

I want to sample from the distribution of a mixture distribution. The hierarchical model is $x_i\sim f$, where: $$f(x\mid \theta_1,\dots,\theta_p, w_1,\dots,\omega_p) = \sum_{j=1}w_p\varphi(x\mid\...
0
votes
0answers
24 views

Shape of parameters marginal posterior in hierarchical Bayes model

Consider a generic hierarchical Bayes model with data $y_i\sim p(y|\theta_i)$, dependent of parameters $\theta_i\sim p(\theta|\phi)$ and hyperparameters $\phi\sim p(\phi)$. Furthermore, assume that $\...
3
votes
1answer
53 views

Auxiliary variable Gibbs sampler

Suppose we want to sample from a pdf $f(x_1,x_2)$. It's easy to sample from $x_1 \vert x_2$, but not $x_2 \vert x_1$, so we introduce an auxiliary variable $u$ such that $\int f(x_1,x_2,u) du = f(x_1,...
0
votes
1answer
50 views

Clusters keep switching in Gibbs sampling of Dirichlet Process Mixture Model

All the code and data for this question is on GitHub (stackexchange.R script). I've got multivariate Bernoulli data that I'd like to analyse using Bayesian Mixture ...
0
votes
1answer
32 views

Inference of the collapsed Gibbs sampling for LDA

I am trying to understand the inference procedure of collapsed Gibbs sampling for LDA model. I refer to this document and LDA wiki page. I cannot figure out how does it simplify the sample equation ...
1
vote
2answers
147 views

Gibbs algorithm using negative binomial produces NAs

I have the following full conditionals distributions: $$ X_2|X_1=x_1\sim Bin(x_1,p)\\ X_1|X_2=x_2\sim NegBin(x_2,p) $$ So I'm using the following code to generate a sample from each one: ...
0
votes
0answers
41 views

Can Gibbs Sampling be used to generate synthetic data from the posterior distribution?

I am working with a few datasets. I would like to expand some of them somehow by creating synthetic data. Is it possible to use a Gibbs Sampler to, by sampling a given distribution of the original ...
0
votes
0answers
51 views

Metropolis-within-Gibbs for parametric inference of a regressive model

I have a regressive model of this form \begin{equation} y=f(\theta)+\varepsilon \end{equation} to describe observations $y$, with noise $\varepsilon$ and a parametric function $f$ with parameters $\...
0
votes
1answer
27 views

Does Gibbs sampling require to know the partition function?

As the title suggest, does gibbs sampling require to know the partition function? For example, if I want to sample variable $a$ and I have worked out $p(a|rest) \propto f(a|rest)$ where $rest$ ...
0
votes
0answers
25 views

Conjugate prior for DPGMMs using Gibbs sampling

I am using Gibbs sampling to infer DPGMMs. The prior for multivariate Gaussians is Normal-inverse Wishart. But it turns out that the covariances are not estimated accurately. Here is codes and results....
2
votes
0answers
35 views

FFBS algorithm for estimating mean log-return parameter in stochastic volatility jump model

I am currently attempting to replicate this model: https://arxiv.org/pdf/1809.01501.pdf in r. My (first) problem is regarding how to sample from conditional posterior for mu, $(μ_{(j)}|Y_n, J_{(j−1)}...
2
votes
1answer
76 views

What's the role of the data in Gibbs Sampling?

Trying to wrap my mind around Gibbs Sampling. Across many answers in this same forum, I constantly notice that the examples shown do not actually require an observed data set (First example (with R ...
0
votes
0answers
25 views

precision or variance of a Gamma distribution in a Gibbs Sampler?

I want to confirm my thinking on a quick question I have regarding the Normal-Gamma Gibbs sampler that we see so often, but I am unable to find a satisfactory answer. If we are interested in ...
8
votes
1answer
77 views

What is the correct way to write the elastic net?

I am confused about the correct way to write the elastic net. After reading some research papers there seems to be three forms 1) $\exp\{-\lambda_1|\beta_k|-\lambda_2\beta_k^2\}$ 2) $\exp\{-\frac{(\...
0
votes
0answers
25 views

Gibbs sampling for drawing samples and estimating parameters

I'm learning Bayesian inference by myself and having a difficulty for understanding Gibbs sampling. From what I understood, Gibbs sampling is to draw samples from a given probability distribution $p(...
0
votes
0answers
63 views

Gibbs sampler: how can thinning equal to the number of iterations work?

I fit an LDA topic model, using the R package topicmodels. No hiccups and everything runs smoothly, my question here is conceptual. When controlling the Gibbs sampler, the default value (in the ...
0
votes
1answer
33 views

Direct/indirect sampling of conditionals in Gibbs sampling

I have some problems understanding the definition of Gibbs sampling. Let us take into consideration a bivariate distribution \begin{equation} \pi(x_1,x_2): S \subset \mathcal{R^2} \rightarrow \...
2
votes
0answers
307 views

An efficient way to generate multivariate normal distribution by Gibbs sampler? [closed]

When learning Gibbs sampler, the most used example is bivariate normal. But what if we want to simulate multivariate normal distribution? The computation (mean and variance) of conditional ...
3
votes
0answers
42 views

antagonistic simulated annealing

Simulated annealing aims at a series of target distributions $$\pi_T(x)\propto\exp\{T\,H(x)\}$$ to find the maximum of the function $H$ and its argument $$\arg_x\max_{x\in \mathfrak X} H(x)$$ if the ...
2
votes
0answers
59 views

How to implement a M-H step in a Gibbs sampling

I am having trouble implementing a Metropolis Hastings step in a Gibbs sampling problem. The following code was taken from https://www.stat.colostate.edu/computationalstatistics/ Details: It is a ...
2
votes
1answer
94 views

Gibbs sampler for a multilevel model with no predictors in R

I'm working on multilevel models and want to know how they are estimated in R. For that purpose I'm reading amongst other things "Data Analysis Using Regression and Multilevel/Hierarchical Models" by ...
0
votes
0answers
27 views

Gibbs sampling where I can only find the mode of conditionals?

I'm trying to solve a problem with Gibbs Sampling, so I'm trying to do: $$ x_1^1 \sim p(x_1 | x_2^0, x_3^0)\\ x_2^1 \sim p(x_2 | x_1^1, x_3^0)\\ x_3^1 \sim p(x_3 | x_1^1, x_2^1)\\ x_1^2 \sim p(x_1 | ...
1
vote
0answers
31 views

Finding the posterior distribution of mean and variance given data sample using Gibbs Sampling?

I have the following hierachical bayesian model - $\mathbf{x}|\mathbf{c},\sigma^2 \sim \mathcal{N}(\mathbf{x}|\mathbf{c},\sigma^2)$ $\mathbf{c}|\mathbf{c}_1,\sigma^2_2 \sim \mathcal{N}(\mathbf{c}|\...
8
votes
1answer
174 views

Would an “importance Gibbs” sampling method work?

I suspect this is a fairly unusual and exploratory question, so please bear with me. I am wondering if one could apply the idea of importance sampling to Gibbs sampling. Here's what I mean: in Gibbs ...
0
votes
0answers
21 views

Determine frequent states

I have a distribution from which I can sample (namely, a Boltzmann Machine). Which methods exists to determine frequent states (states with high probability) / the most frequent state (state with the ...
0
votes
0answers
37 views

forward sampling for Bayesian network with continuous variables and equation-based causal relationships

I have a physical system which can be represented by the following Bayesian network. It has the following characteristics 1) The encoded variables are continuous variables 2) The causal ...
0
votes
0answers
94 views

Gibbs sampler for ARIMA AR(1) parameters: division by zero

Suppose the following AR(1) model: $$ y_t = \mu + \phi (y_{t-1} - \mu) + \epsilon_t $$ with $\epsilon_t \sim \mathcal{N}(0,\sigma^2)$. Following issue arises when sampling from $P(\mu_i \;|\; \phi_{...
6
votes
3answers
250 views

Conditional distribution of $\exp(-|x|-|y|-a \cdot |x-y|)$

I am trying to use Gibbs sampling or Metropolis-Hastings to draw samples from the joint distribution$$f(x,y)\propto\exp(-|x|-|y|-a \cdot |x-y|)$$ For this I need the conditional distributions of $x$ ...
2
votes
1answer
43 views

Gibbs sampling allocations for time dependent observations from this model

I observe $N$ observations $\{x_{1,t_1}, \dots, x_{N,t_N}\}$ from a $k$ component Gaussian Mixture model. The $i$th observation is seen at time stamp $t_i$ and is distributed such that each $x_{i,t_i}|...
1
vote
0answers
24 views

Gibbs Sampler for mixture models: shall I skip some samples to avoid to use correlated samples? [duplicate]

I am implementing a Gibbs sampler in order to estimate the parameters of a mixture model. Assuming that the parameters are contained in a vector $\theta$ what I will do is: Implement and run the ...
0
votes
0answers
51 views

bayesian decision making - comparing expected loss

The problem is like this: Suppose that I am considering which country should I invest on, country A and country B, based on their GDP growth rate $\alpha$. There are two possible choices for each ...
-1
votes
1answer
270 views

Inferring GMM parameters with Gibbs Sampling

On my book, "Machine Learning A Probabilistic Approach". It's stated that is straightforward to derive a Gibbs sampling algorithm to fit a mixture model, especially if we use conjugate priors. So ...