Questions tagged [gibbs]

The Gibbs sampler is a simple form of Markov Chain Monte Carlo simulation, widely used in Bayesian statistics, based on sampling from full conditional distributions for each variable or group of variables. The name comes from the method being first used on Gibbs random fields modeling of images by Geman and Geman (1984).

Filter by
Sorted by
Tagged with
1
vote
1answer
126 views

How to Gibbs sample proportional to a probability

I am reading this tutorial on Hierarchical Chinese Restaurant Process. On pdf page 141 (slide title: MCMC Problem Specification for N-grams) it says: $$F(s_{1,k})=\frac{\alpha^{S'_1+s_{1,k}}}{(\alpha)...
1
vote
1answer
82 views

Gibbs sampling for mixed variables [duplicate]

Let X a continous variable and Y a binary variable with joint distribution : $$p(x,y;\beta,\rho_1,\rho_2,\phi_1,\phi_2)=\frac{1}{Z(\beta,\rho_1,\rho_2,\phi_1,\phi_2)}\exp(-0.5 \beta x^2+1_{y=0}\rho_1 ...
1
vote
1answer
81 views

Deriving Gibbs sampler for specific mixture model

Let $\theta_i$ be an indicator which is $0$ if score, $X_i$, is the same for both opponents, $1$ if different: $X_i|\theta_i \stackrel{\text{ind}}{\sim} (1-\theta_i) U(0, 1) + \theta_i Beta(1, \...
1
vote
1answer
59 views

Is it possible for iterations to spike in Gibbs sampling?

After performing Gibbs sampling, I looked at a trace plot for one of my parameters and it appeared to spike at certain points. Is this possible or is it likely that I just coded my sampler wrong?
1
vote
1answer
109 views

PyMC consistently under estimating results found in paper. Possibly not sampling enough?

I have been trying to build confidence in (my ability to correctly use) PyMC by working examples. Namely, I have been working on Chickering and Pearl 1997, and more specifically on their 'artificial' ...
1
vote
2answers
915 views

Any good book for learning probability programming

Are there any good books for me to learn probability programming? For example, I am new to Latent Dirichlet allocation (LDA) and Gibbs sampling. I have read some books about the techniques, but it ...
1
vote
1answer
539 views

Computing conditional expectation of ordered normal random variables

There are $m$ normally distributed, independent random variables $N_1, \ldots, N_m$ with distinct means $\mu_1, \ldots \mu_m$ and standard deviations $\sigma_1, \ldots, \sigma_m$. Then, we observe a ...
1
vote
1answer
35 views

Why iterations of Gibbs sampling for a bivariate Gaussian distribution can be seen as random walk?

In Section 4.4 of the excellent technical report Probabilistic Inference using Markov Chain Monte Carlo Methods, the author tries to analyze the performance of Gibbs and Metropolis algorithm with ...
1
vote
0answers
16 views

Plotting a random walk on R [closed]

I've run a Gibbs sampler and obtained a sample for $X_1$ and $X_2$. I'm trying to recreate a plot like this one: How do I recreate the walk part on R?
1
vote
0answers
20 views

Particle Gibbs Sampler For Regime-Switching Nonlinear Gaussian SSM

I'm reading this paper on using a non-linear Gaussian SSM for measuring regime-switching leverage effect using stock market data. I'm using it as jump-off point for an undergraduate paper. My advisor ...
1
vote
0answers
12 views

Gibbs sampling Bayesian conditional distribution for mean of a Normal distribution

first post here in CV. I'm currently working on a textbook exercise on Gibbs Sampling and got stuck on naming the distribution for one of the conditional distributions. Question Consider a normal ...
1
vote
0answers
42 views

Gibbs sampling proposals for bivariate normal?

I'm very familiar with Metropolis-Hastings, having implemented the algorithm myself to handle "toy problems." Gibbs sampling, however, is a bit trickier for me as I'm not quite certain what ...
1
vote
0answers
18 views

How does pymc3 posterior simulation work in this simple case without having the full conditional distributions?

I'm trying to estimate the posterior distribution of the gamma parameters alpha and beta given that my data comes from a gamma distribution and the priors I chose come from two uniform distributions. ...
1
vote
0answers
37 views

parameter estimation on the LDA model

I have a problem with estimating the parameters of $\theta$, and $\phi$ in the Latent Dirichlet Allocation (LDA) model. The article Finding scientific topics has done the estimation of the parameters ...
1
vote
1answer
43 views

Relationship between variational inference and sampling in a Boltmzann-machine-like network

In this paper concerning a Boltzmann-machine-like network and its variational mean field approximation, the authors write In the stochastic system as well as the deterministic system, units evolve ...
1
vote
0answers
19 views

Gibbs sampler with adaptive linear transformation

It is a well known fact that linear transformations can dramatically improve the performances of a Gibbs sampler when a ridge-like joint likelihood function occurs. Can I make an algorithm that ...
1
vote
0answers
37 views

Sampling states of an “unnatural” Hamiltonian System

I would like to sample from a Gibbs distribution given by $$f(p, q) = \frac{1}{\mathcal{Z}}e^{-H(p, q; \omega, J)}$$ where $H$ is the Hamiltonian on generalized coordinates $(p,q)\in \mathbb{R}^{2n}...
1
vote
0answers
22 views

Why the nodes in a Boltzmann machine need to be sampled one at a time?

Typically, we use Gibbs sampling to update (or generate samples from) energy based models. This means we update each node while keeping its markov blanket constant. Why can't we update/sample all ...
1
vote
0answers
227 views

Bayesian Gamma Regression Update

I'm looking for a resource that explains how to do update the coefficients for a Bayesian gamma regression using Gibbs sampling. Specifically, if $y_i \sim Gamma(\alpha,\beta_i)$ and my data ...
1
vote
0answers
19 views

Markov chain Monte Carlo, Mixing Time

How do you estimate the mixing time for a markov chain? I read somewhere one can use the sum of the auto-correlation coefficients or the sum of the auto-covariance coefficients, but I cannot seem to ...
1
vote
0answers
46 views

What's the point of Gibbs Sampling? [duplicate]

I am reading a book on doing Bayesian Data Analysis. I have just learned what the Metropolis Hastings (MH) Algo does, at least in relation to Bayesian Data Analysis. My understanding of the MH Algo ...
1
vote
1answer
28 views

Assign an error to the parameters of MAP estimate

Through a MCMC Gibbs sampler I obtain $M$ chains of the parameters vector $\mathbf{\theta}$, meaning that each component of $\mathbf{\theta}$ is the value of one parameter at a given iteration. ...
1
vote
0answers
30 views

Is there something wrong with my Bayesian hierarchical spatio-temporal model?

I built a Bayesian spatio-temporal model and one of the parameters to be estimated is the random spatial effects s. The random spatial effect is assigned an intrinsic conditional autoregressive prior (...
1
vote
0answers
67 views

Understand the Holmes and Held (2006) Bayesian probit MCMC algorithm

Holmes and Held (2006) suggest a simple approach to reduce autocorrelation in the MCMC algorithm proposed by Albert and Chib (1993). HH (2006) propose to update $\beta$ and $z$ jointly, making use of ...
1
vote
0answers
40 views

Finding the posterior distribution of mean and variance given data sample using Gibbs Sampling?

I have the following hierachical bayesian model - $\mathbf{x}|\mathbf{c},\sigma^2 \sim \mathcal{N}(\mathbf{x}|\mathbf{c},\sigma^2)$ $\mathbf{c}|\mathbf{c}_1,\sigma^2_2 \sim \mathcal{N}(\mathbf{c}|\...
1
vote
0answers
27 views

Gibbs Sampler for mixture models: shall I skip some samples to avoid to use correlated samples? [duplicate]

I am implementing a Gibbs sampler in order to estimate the parameters of a mixture model. Assuming that the parameters are contained in a vector $\theta$ what I will do is: Implement and run the ...
1
vote
0answers
47 views

How exactly does Gibbs sampling work in Markov Networks?

I was going through the Probabilistic Graphical Modelling course by Stanford and they used a network such as this one-https://imgur.com/gallery/k0C8FY2 Now if we want to sample P(A|B), how would we ...
1
vote
0answers
60 views

HDP: Gibbs sampler implementation

I am trying to recreate the model proposed by Gao et al. (2011), based on the Hierarchical Dirichlet Process proposed by Teh and al. (2005). To estimate the model (let's call it iHDP) I need to ...
1
vote
0answers
77 views

Non-Identifiable Multivariate Normal Posterior

So I have a theoretical question about what looks like, in my opinion, a multivariate normal distribution. The issue comes with the fact that the data is distributed with likelihood: Y |θ1, θ2 ∼ N(θ1 ...
1
vote
0answers
59 views

help interpreting plot of MCMC sample

I am estimating a model using MCMC (Gibbs Sampling). Because of the complexity of the model, I have been running two chains with many iterations. A plot of the draws for each parameter reveals a ...
1
vote
0answers
149 views

How to know whether a Gibbs sampler is irreducible? [duplicate]

How to know whether a Gibbs sampler is irreducible? I know that the Gibbs sampler in e.g. two variable case constructs a sequence of r.v.s $(X_1^{(i)}, X_2^{(i)})$ by sampling from the related ...
1
vote
0answers
22 views

Averaging Across Gibbs Sampling Runs with Reduced Dimensions

I need help thinking through my approach to Gibbs Sampling of many parameters and I'd like to know if there is literature on this topic: I have a dataset with 3 dimensions: ...
1
vote
0answers
35 views

Gibbs sampling with expectations instead of sampling

I see there is something called Iterated Conditional Models (ICM), which is a sort of Gibbs sampling where, instead of sampling, we use the value that maximizes the conditional. That is: ...
1
vote
0answers
280 views

Collapsed Gibbs sampler on Hierarchical Dirichlet Process Mixture Model

I am trying to design a collapsed Gibbs sampler on a mixture model based on Hierarchical Dirichlet Process ($g\sim DP(\gamma, b)$, $\pi\sim DP(\alpha, g)$ ). Should I resample from the posterior of ...
1
vote
0answers
262 views

Get different results with different sampling order in Gibbs sampling: what could be wrong?

In sampling a complex spatio-temporal model by Gibbs sampling, I found if I change the order of sampling (for example, to sample $P(\theta_1,\theta_2|D)$, in one try, I sample $\theta_1\sim P(\theta_1|...
1
vote
0answers
117 views

Gibbs Sampling / Monte Carlo with Weights

Consider pairs of data and their population weights $(y_i, \omega_i), i = 1, 2, \dots$ alongside some hierarchical structure, $$y_i\leftarrow\theta_{i(j)} \leftarrow \gamma$$ where $i(j)$ is perhaps ...
1
vote
1answer
66 views

Gibbs sampler -transformation of conditional posterior

If my conditional posterior $\pi(\sigma^{-2}|\mathbf{y },\mu)\sim Gamma(a,b)$, how can I get the conditional posterior $\pi(\sigma^{2}|\mathbf{y },\mu)$ with a transformation? The reason I ask is ...
1
vote
0answers
31 views

Sampling from a truncated random effects distribution

How would one sample observations $T_{ij} = U_i + \varepsilon_{ij}$, where the distribution of $U_i, \varepsilon_{ij}$ are known and mutually independent, condition on the fact that $L_{ij} \le T_{ij} ...
1
vote
0answers
85 views

Methods to solve a compound distribution MLE I can't write in closed form

What are some methods of solving a MLE parameter estimate where the likelihood function can't be written in closed form? I have a compound distribution that I don't have a closed form for (no ...
1
vote
0answers
55 views

Gibbs sampling confusion

I'm just wondering if i'm doing this process correctly i'm a little confused as to the answers i'm getting: I have $P(x=0,y=0) = P(x=1,y=1) = 0.5$ and $P(x=0,y=1) = P(x=1,y=0) = 0$ I calculated $P(...
1
vote
0answers
940 views

How to check the convergence in the collapsed Gibbs sampling of LDA? [closed]

I am trying to implement the LDA model fit by collapsed Gibbs sampling by myself. I have go through this article. And there is a clear pseudo code (section 5.5), ...
1
vote
0answers
237 views

How to sample random variables (x,y) from a bivariate Cauchy distribution using a Gibbs sampler?

A bivariate Cauchy distribution is equivalent to a bivariate t-distribution with 1 degree of freedom.
1
vote
0answers
94 views

How does one determine which variables can be collapsed in Gibbs Sampling?

I am going through the derivation for Gibbs Sampling update equations for LDA. The claim is that $\theta_{d}$ (document specific topic distribution) and $\phi_k$ the topic-word distribution can be ...
1
vote
0answers
95 views

Gibbs within Collapsed Gibbs?

I have a model with variables $X_{1}, X_{2}, X_{3}, X_{4}$. I would like to sample it within a larger MCMC chain using: $(X_{1}, X_{2}) \sim P(X_{1}, X_{2})$ $(X_{3}, X_{4}) \sim P(X_{3}, X_{4} \mid ...
1
vote
0answers
285 views

plot log likelihood function evolution in mcmc simulations

Is it possible to plot log likelihood function evolution in mcmc simulations? I have a mixture model and its parameters are estimated using the gibbs sampling method in r environment and using the ...
1
vote
0answers
167 views

Detecting Gibbs Sampler convergence with Raftery and Lewis Diagnostic

Hi! I'm trying to understand and implement the Raftery and Lewis Diagnostic for detecting the number of iterations required for a gibbs sampler but cant seem to understand the formula. Can someone ...
1
vote
0answers
191 views

How do I check or validate the RBM (Restricted Boltzmann Machine) Model?

I'm trying to implement RBM, then i used play tennis case to test the rbm. I've tried autoencoder before, and the result was good. Actually, I confuse with the function of RBM it self, i think it ...
1
vote
0answers
54 views

Efficient generation of graph structured correlated random variables via MCMC/Gibbs

Sometime back I had asked this question about generating correlated random draws based on the correlation structure given by a graph. Link Here The solution there requires to create $n\times n$ ...
1
vote
0answers
80 views

How can I sample multivariate binary variables such that sum of them follows a gamma distribution?

Edit: Since the original question was confusing as whuber pointed out, let me rephrase the question with a Poisson distribution instead of a gamma distribution. The energy term of a Poisson ...
1
vote
2answers
84 views

Sampling from a portion of the normal distribution?

I have a a conditional distribution $p(X_1 | \theta) \propto MVN(\mu, \Omega) \pi(X_1)$ where $X_1=[x_1, x_2, \dots, x_n]'$ and $\pi(X_1)=1$ when all $x_i \in [0,a)$ and $0$ otherwise. Is there any ...

1
4 5
6
7 8