Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [gibbs]

The Gibbs sampler is a simple form of Markov Chain Monte Carlo simulation, widely used in Bayesian statistics, based on sampling from full conditional distributions for each variable or group of variables. The name comes from the method being first used on Gibbs random fields modeling of images by Geman and Geman (1984).

0
votes
0answers
20 views

Gibbs sampling where I can only find the mode of conditionals?

I'm trying to solve a problem with Gibbs Sampling, so I'm trying to do: $$ x_1^1 \sim p(x_1 | x_2^0, x_3^0)\\ x_2^1 \sim p(x_2 | x_1^1, x_3^0)\\ x_3^1 \sim p(x_3 | x_1^1, x_2^1)\\ x_1^2 \sim p(x_1 | ...
1
vote
0answers
18 views

Finding the posterior distribution of mean and variance given data sample using Gibbs Sampling?

I have the following hierachical bayesian model - $\mathbf{x}|\mathbf{c},\sigma^2 \sim \mathcal{N}(\mathbf{x}|\mathbf{c},\sigma^2)$ $\mathbf{c}|\mathbf{c}_1,\sigma^2_2 \sim \mathcal{N}(\mathbf{c}|\...
5
votes
0answers
58 views

Would an “importance Gibbs” sampling method work?

I suspect this is a fairly unusual and exploratory question, so please bear with me. I am wondering if one could apply the idea of importance sampling to Gibbs sampling. Here's what I mean: in Gibbs ...
0
votes
0answers
19 views

Determine frequent states

I have a distribution from which I can sample (namely, a Boltzmann Machine). Which methods exists to determine frequent states (states with high probability) / the most frequent state (state with the ...
0
votes
0answers
9 views

forward sampling for Bayesian network with continuous variables and equation-based causal relationships

I have a physical system which can be represented by the following Bayesian network. It has the following characteristics 1) The encoded variables are continuous variables 2) The causal ...
0
votes
0answers
68 views

Gibbs sampler for ARIMA AR(1) parameters: division by zero

Suppose the following AR(1) model: $$ y_t = \mu + \phi (y_{t-1} - \mu) + \epsilon_t $$ with $\epsilon_t \sim \mathcal{N}(0,\sigma^2)$. Following issue arises when sampling from $P(\mu_i \;|\; \phi_{...
3
votes
2answers
153 views

Conditional distribution of $\exp(-|x|-|y|-a \cdot |x-y|)$

I am trying to use Gibbs sampling or Metropolis-Hastings to draw samples from the joint distribution$$f(x,y)\propto\exp(-|x|-|y|-a \cdot |x-y|)$$ For this I need the conditional distributions of $x$ ...
2
votes
1answer
33 views

Gibbs sampling allocations for time dependent observations from this model

I observe $N$ observations $\{x_{1,t_1}, \dots, x_{N,t_N}\}$ from a $k$ component Gaussian Mixture model. The $i$th observation is seen at time stamp $t_i$ and is distributed such that each $x_{i,t_i}|...
1
vote
0answers
23 views

Gibbs Sampler for mixture models: shall I skip some samples to avoid to use correlated samples? [duplicate]

I am implementing a Gibbs sampler in order to estimate the parameters of a mixture model. Assuming that the parameters are contained in a vector $\theta$ what I will do is: Implement and run the ...
0
votes
0answers
28 views

bayesian decision making - comparing expected loss

The problem is like this: Suppose that I am considering which country should I invest on, country A and country B, based on their GDP growth rate $\alpha$. There are two possible choices for each ...
0
votes
1answer
23 views

Inferring GMM parameters with Gibbs Sampling

On my book, "Machine Learning A Probabilistic Approach". It's stated that is straightforward to derive a Gibbs sampling algorithm to fit a mixture model, especially if we use conjugate priors. So ...
1
vote
1answer
70 views

Conditional distribution in this Gaussian Mixture Model

Say I observe $N$ observations $\{x_1, \dots, x_N\}$ from a $k$ component Gaussian Mixture model, with $k > 0$ known and is such that each $x_i|\boldsymbol{\pi}, \boldsymbol{\mu} \sim \sum_{j=1}^{k}...
1
vote
1answer
20 views

Gibbs sampler for Dirichlet Process concentration parameter

I am trying to implement a Gibbs sampler for Hierarchical Dirichlet process, but I cannot seem to correctly estimate the concentration parameters. I therefore started testing just this part of a ...
2
votes
1answer
82 views

Conditional Distribution to sample

Suppose I have six data points (n,x): (14,5), (13,4), (7,3), (10,5), (12,7), (20,13) which are realizations of binomial experiments on n trials with x successes respectively.. And I assume I ...
11
votes
2answers
354 views

Is Gibbs sampling an MCMC method?

As far as I understand it, it is (at least, that is how Wikipedia defines it). But I've found this statement by Efron* (emphasis added): Markov chain Monte Carlo (MCMC) is the great success story ...
0
votes
1answer
146 views

How to create a distribution and sample?

Suppose we are given some small set of data on bundles of electrical wires and increasing voltages run through them, and we note how many of the individual wires fail. So for example, a large data ...
1
vote
0answers
21 views

How exactly does Gibbs sampling work in Markov Networks?

I was going through the Probabilistic Graphical Modelling course by Stanford and they used a network such as this one-https://imgur.com/gallery/k0C8FY2 Now if we want to sample P(A|B), how would we ...
1
vote
1answer
176 views

Conditional distribution for Gibbs sampling for Gaussian mixture

If we draw $n$ i.i.d. points $x_1,x_2,\dots,x_n$ from the following Gaussian mixture: $$ \frac 12 \mathcal N(x \mid \mu_1,1) + \frac 12 \mathcal N(x\mid \mu_2,1) $$ and the prior $p(\mu_1 , \mu_2 )$ ...
1
vote
1answer
39 views

Sampling from partial posterior

I'm reading a paper where the authors have something like the following as one step in their MCMC: $$ \begin{align} y&=\rho_1 x_1+\rho_2 x_2+\epsilon\\ z&=\beta\tilde{y}+u \end{align} $$ ...
1
vote
0answers
35 views

HDP: Gibbs sampler implementation

I am trying to recreate the model proposed by Gao et al. (2011), based on the Hierarchical Dirichlet Process proposed by Teh and al. (2005). To estimate the model (let's call it iHDP) I need to ...
2
votes
0answers
77 views

Can Bayesian Optimization solve this problem?

Suppose ${\bf{x}} = (x_1,\ldots,x_n)$ and $f({\bf{x}})\propto 1_A({\bf{x}}) \prod_{i=1}^n {x_i}^{\alpha_i-1} e^{-\beta_i x_i}$ , i.e. $f$ is proportional to the product of independent gamma ...
4
votes
1answer
37 views

Find joint maximum of a sampled density

I used a sampling method to fit a model with three parameters to data, by supplying the likelihood function and priors. (I'm using JAGS but I think this applies to any method). I obtain triplets of ...
0
votes
1answer
189 views

What is the difference between monte carlo integration and gibbs sampling?

I am aware that both are methods of sampling from the posterior. MC integration replaces the integral by a sample MC sample. Is this sample independent? Gibbs sampling is a class of MCMC ...
2
votes
1answer
28 views

Slice Sampling asks to draw from $f^{-1}]y,+\infty[$

Slice Sampling asks to draw uniformly from $f^{-1}]y,+\infty[$. Wikipedia page However, how can we be sure that a uniform defined over the set $f^{-1}]y,+\infty[$ is in fact proper? If I had to ...
3
votes
1answer
130 views

Importance weight of conditioned particle in conditional SMC

In a generic particle filter, I understand the importance weights for each particle are calculated as $w_t^s \propto w_{t-1}^s \frac{p(y_t \mid z_t^s) p(z_t^s \mid z_{t-1}^s)}{q(z_t^s \mid z_{t-1}^s, ...
0
votes
0answers
21 views

Markov Chain Monte Carlo [duplicate]

what are the differences between M-H algorithm and M-H-within-Gibbs algorithm. If possible, upload for me the two algorithms please.
14
votes
1answer
855 views

Stan $\hat{R}$ versus Gelman-Rubin $\hat{R}$ definition

I was going through the Stan documentation which can be downloaded from here. I was particularly interested in their implementation of the Gelman-Rubin diagnostic. The original paper Gelman & ...
3
votes
1answer
50 views

Sampling posterior of empty cluster in GMM and Gibbs

Consider performing inference via a standard Gibbs sampler for a standard Gaussian Mixture Model (GMM) with $k$ components that are Gaussians $$\mathcal{N}(\mu_{k}, \sigma^{2}_{k})$$ where we assume ...
0
votes
1answer
27 views

Distribution sampling when no analytic expression

The goal of gibbs sampling is to sample the joint distribution when this latter has not an analytic expression, by deriving the conditional distribution of each variable. So it is supposed that the ...
4
votes
2answers
260 views

Posterior computation for Laplace distribution

I am dealing with being Bayesian and looking for a closed form for a posterior for the scale parameter $\tau$ of a Laplace distribution, such that I can derive a full conditional in my Gibbs sampler. ...
2
votes
2answers
64 views

Generating samples from Gibbs method

I have a following homework in a subject called "Monte-Carlo Methods". I would be very thankful, if you could help me with this one, because I'm a bit stuck with this one .. The task is as follows:...
3
votes
1answer
54 views

Gibbs Sampling vs. Using Raw Probability in Contrastive Divergence

In Hinton's Practical Guide to Training Restricted Boltzmann Machines, Section 3, he discusses different situations in which one should take a sample from the Gibbs sampling process, and other ...
2
votes
0answers
65 views

Bayesian prior and posterior computation for a truncated normal

I have to deal with data in a Bayesian framework, ultimately devising a Gibbs sampler for inferring all my distributions parameters. Specifically, suppose I observe some univariate data distributed ...
1
vote
0answers
43 views

Non-Identifiable Multivariate Normal Posterior

So I have a theoretical question about what looks like, in my opinion, a multivariate normal distribution. The issue comes with the fact that the data is distributed with likelihood: Y |θ1, θ2 ∼ N(θ1 ...
2
votes
1answer
127 views

Why can Gibbs sampling outputs be used in Rao-Blackwellization?

I'm currently learning Chib (1995)'s method to calculate the marginal likelihood of a Bayesian model using Gibbs sampling outputs. I'm stuck in the Rao-Blackwellization step. Suppose $\mu$ and $\phi$...
0
votes
0answers
34 views

A conditioning simplification in a graphical clustering model

The graphical model associated with this problem is (reproduction from Murphy's Machine Learning a Probabilistic Perspective) and the author argues on page 843 of MLAPP, Intuitively, it is right. ...
3
votes
0answers
129 views

Is burn-in necessary for MCMC/Gibbs sampling if I have samples from the true distribution already?

Say I have some samples from a distribution $p$, and I want to get more samples using MCMC/Gibbs sampling. Since the existing samples are known from the equilibrium distribution $p$, if I use them as ...
2
votes
1answer
65 views

Why does $P(\theta_1\mid D, \theta_2) \propto P(D \mid \theta_1, \theta_2)P(\theta_1)$ hold?

Suppose that in a Bayesian framework we have observed data $D$, using independent prior distributions on the parameters of the model, denoted by $\theta_1, \theta_2$. Then, the joint posterior ...
0
votes
1answer
64 views

prior for initial values of Kalman Filter

I'm studying Carter and Kohn's (1994) implementation of the Gibbs sampler for Bayesian analysis of state space models. In their paper, they assume the starting value, call it $\beta_0$, of the state ...
5
votes
1answer
67 views

How to use Gibbs sampling when target function is known only up to normalising constant?

Assume we want to use Gibbs sampling to get an estimate of the parameter , and that we have the following expression for the conditional posterior of : If I am not mistaken, this means that the ...
1
vote
0answers
52 views

help interpreting plot of MCMC sample

I am estimating a model using MCMC (Gibbs Sampling). Because of the complexity of the model, I have been running two chains with many iterations. A plot of the draws for each parameter reveals a ...
0
votes
0answers
19 views

What is the meaning of the clique-function when factorising undirected graphs?

I am currently reading Graphical Models and Message-Passing Algorithms: Some Introductory Lectures and am quite at the beginning. I understand the factorization for directed graphical methods, which ...
3
votes
2answers
208 views

Gibbs sampling and Bayesian inference

I wanted to get a more in-depth understanding of sampling algorithms, and so I thought I could start with the very simple example of a binomial or bernoulli likelihood with a beta prior (since it is ...
1
vote
0answers
61 views

Burning and Sampling in Gibbs sampler [closed]

I am confused about the concept of sampling in the Gibbs sampler after the burn-in loops. This is the basic problem, I have a picture composed of 1's and 0's. This is a noisy version and I am trying ...
1
vote
1answer
242 views

When can the collapsed Gibbs sampler be implemented?

I understand Gibbs sampling is a means of statistics inference, and it seems that sometimes certain variables can be integrated out in the sampling process, known as collapsed Gibbs sampling. I really ...
1
vote
0answers
62 views

How to know whether a Gibbs sampler is irreducible? [duplicate]

How to know whether a Gibbs sampler is irreducible? I know that the Gibbs sampler in e.g. two variable case constructs a sequence of r.v.s $(X_1^{(i)}, X_2^{(i)})$ by sampling from the related ...
0
votes
2answers
121 views

How is the accuracy of the results of Gibbs sampler measured?

How is the accuracy of the results of Gibbs sampler measured? Most resources merely say to iterate it $k$ times. But how does one infer the accuracy of the result?
5
votes
1answer
928 views

Metropolis-Within-Gibbs sampling with only marginal distribution known for a subset of variables

Typically in Gibbs sampling we want to sample from a joint distribution $p(X_1, X_2, ..., X_N)$, but because the joint is hard to sample from directly, we instead achieve this by iteratively sampling ...
1
vote
0answers
128 views

Missing data in Gibbs sampling for dynamic linear models

Suppose I have the following DLM: $x_t = \Phi x_{t-1} + w_t$ $y_t = A x_t + v_t$ $x_0 \sim N(\mu_0,\Sigma_0)$ $w_t \sim N(0,Q)$ $v_t \sim N(0,R)$ Let $\Theta = \{\mu_0,\Sigma_0,\Phi,Q,A,R\}$. I ...
4
votes
1answer
831 views

Explanation regarding Gibbs Sampling

I am new to MCMC and reading a intro paper regarding Gibbs sampling. However, there are two parts in the paper I cannot understand and get stuck. The first part is equation 2.3 in page 168. It says ...