Questions tagged [gibbs]

The Gibbs sampler is a simple form of Markov Chain Monte Carlo simulation, widely used in Bayesian statistics, based on sampling from full conditional distributions for each variable or group of variables. The name comes from the method being first used on Gibbs random fields modeling of images by Geman and Geman (1984).

135 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
6
votes
0answers
1k views

Is my OpenBUGS / WinBUGS model well specified?

I've just started trying to use OpenBUGS for Bayesian analysis of stochastic volatility models. In particular, I'm trying to calculate stochastic covariance, similar to the DC-MSV model specified by ...
5
votes
0answers
678 views

Gibbs sampling deriving complete conditionals with mixture priors

My question is about the derivation of the complete conditionals for Gibbs sampling in a hierarchical model where some of the parameters are mixtures of point-masses and Normal distributions. The ...
5
votes
0answers
339 views

Bayesian estimates for Deming regression coinciding with least-squares estimates

Consider the following Deming model with independent replicates : $$x_{i,j} \mid \theta_{i} \sim {\cal N}(\theta_{i}, \gamma_X^2), \quad y_{i,j} \mid \theta_{i} \sim {\cal N}(\alpha+\beta\theta_{i}, \...
5
votes
0answers
1k views

Gibbs sampling for LDA -- does a small Dirichlet concentration parameter make a difference?

I'm using a Gibbs sampler for Latent Dirichlet allocation as described by Griffiths and Steyvers (http://www.ncbi.nlm.nih.gov/pmc/articles/PMC387300/). The sampling of a new topic $j$ for word $i$ is ...
4
votes
0answers
297 views

Is burn-in necessary for MCMC/Gibbs sampling if I have samples from the true distribution already?

Say I have some samples from a distribution $p$, and I want to get more samples using MCMC/Gibbs sampling. Since the existing samples are known from the equilibrium distribution $p$, if I use them as ...
4
votes
0answers
423 views

Convergence of approximate Gibbs sampling

We have a bivariate random variable $(X,Y)$ for which sampling is challenging. If we were to know how to sample from the conditionals $(X|Y)$ and $(Y|X)$, we could get samples from the joint using ...
4
votes
0answers
284 views

Gibbs sampling for parameter estimation

I am reading the paper by Willemsen et al (2015), "A multivariate Bayesian model for embryonic growth", Statistics in Medicine, 34:8, 1351–1365 where they define the posterior distribution as, \begin{...
4
votes
1answer
174 views

Sampling from multivariate normal conditional on a negative minimum

Let $X\sim \mathcal{N}(\mu,\Sigma)$, where $\mu\in\mathbb{R}^n$ and $\Sigma\in\mathbb{R}^{n\times n}$. How can I efficiently sample from $X | {\min{X}\le 0}$? (I.e. from the distribution of $X$ ...
4
votes
0answers
828 views

Criteria in determining "step size" of Metropolis-hasting algorithms

I am training a complex Bayesian model using Gibbs sampling and Metropolis-Hasting algorithm. Most of the parameters are directly sampled by using conjugate priors except for 3 params which are ...
4
votes
0answers
316 views

When 2% of the Bayesian Model have not converged?

I have model with 20000 latent parameters, set up in a Gibb's sampler. 98% of the parameters and sometimes 99.5% of the parameters satisfy the Geweke convergence statistic, have low autocorrelation ...
4
votes
0answers
267 views

full conditional posteriors for bayesian lasso

I am reading the original Bayesian Lasso paper, and its follow up; They look straightforward to implement, mainly because of the conditional posterior probability for the gibbs sampler; however, I ...
4
votes
0answers
650 views

Gibbs sampling for correlated random variables

Short summary Suppose two latent variables of a hierarchical model are correlated. Let $1-\epsilon$ be the degree of correlation. As $\epsilon\rightarrow 0$ the variables become perfectly correlated ...
4
votes
0answers
409 views

Convergence theorem for Gibbs sampling

The convergence theorem for Gibbs sampling states: Given a random vector $X$ with components $X_1,X_2,...X_K$ and the knowledge about the conditional distribution of $X_k$ we can find the actual ...
4
votes
0answers
5k views

Metropolis-Hastings within Gibbs sampling

Suppose we have the following classical normal linear regression model: $$y_i = \beta_1 x_{1i} + \beta_2x_{2i} + \beta_3x_{3i} + e_i$$ where $e_{i} \sim iid.N(0, \sigma^2)$ for all $i = 1, 2, \cdots,...
4
votes
0answers
315 views

Gibbs sampling from full conditionals

I have the following joint density: $p(x_1,x_2,y_1,y_2) \propto \exp\left(−\left(x_1^2+x_2^2+c_1(y_2-y_1)^2+c_2(y_2-y_1)^4\right)\right)$ Can I use Gibbs sampling to sample from that? How can I get ...
4
votes
0answers
508 views

Bayesian estimation using Gibbs sampling for financial models

I am trying to do Gibbs sampling, from this paper. This is a CIR financial model, I want to do Gibbs on its parameters: $$y(t+{\Delta}^{+})=y(t)+(\alpha-\beta y(t)){\Delta}^{+}+\sigma \sqrt{y(t)}{\...
3
votes
0answers
73 views

antagonistic simulated annealing

Simulated annealing aims at a series of target distributions $$\pi_T(x)\propto\exp\{T\,H(x)\}$$ to find the maximum of the function $H$ and its argument $$\arg_x\max_{x\in \mathfrak X} H(x)$$ if the ...
3
votes
0answers
111 views

How to implement a M-H step in a Gibbs sampling

I am having trouble implementing a Metropolis Hastings step in a Gibbs sampling problem. The following code was taken from https://www.stat.colostate.edu/computationalstatistics/ Details: It is a ...
3
votes
1answer
174 views

Gibbs Sampling vs. Using Raw Probability in Contrastive Divergence

In Hinton's Practical Guide to Training Restricted Boltzmann Machines, Section 3, he discusses different situations in which one should take a sample from the Gibbs sampling process, and other ...
3
votes
0answers
978 views

Bayesian estimation of Dynamic Regression with AR(1) parameters

I would like to draw (Bayesian) inference in a dynamic linear regression with regression parameters following independent AR(1) processes $\beta_{t,i} = \mu_i+\beta_{t-1,i}+w_{t,i}$. However, I ...
3
votes
0answers
456 views

When Gibbs Sampling is fast/slow to converge?

Are there any heuristic/theories showing that on what kind of Bayesian models the convergence of Gibbs sampling is fast/slow? For example, from my limited experience, I feel (may be wrong) when a ...
3
votes
0answers
142 views

Seeking help in Bayesian Mixed Effects Model

I am implementing a Bayesian Mixed Effects model in my research problem. The model is written as, $y_i = X_i(\alpha + \beta_i) + \epsilon_i$, where $i = 1, 2, \ldots, m$ is the index of response, $j = ...
3
votes
0answers
205 views

Derive Marginal Posterior to set up Gibbs-Sampler

I am currently trying to replicate a Hierarchical Model for multivariate returns proposed in the paper Portfolio selection using hierarchical Bayesian analysis and MCMC methods. However, in order to ...
3
votes
0answers
477 views

Why is the posterior the stationary distribution of a Gibbs chain?

I'm having trouble understanding the setup here. I'm following Probabilistic Graphical Models by Koller and Friedman. They say that we wish to generate samples from the posterior distribution $P(\...
3
votes
0answers
903 views

Efficiency in Metropolis Vs Gibbs sampling

I have read that Gibbs sampling is more efficient than Metropolis algorithm. Why? Is this due only to the fact the in Gibbs sampling the acceptance rate is $1$, so that the chain needs fewer ...
3
votes
0answers
2k views

How to understand Gibbs distribution

I have a graph model such as Following the Hammersley–Clifford theorem describes that Markov random fields exhibit a Gibbs distribution with an energy function as follows: $$P(x)=\frac {exp(-U(x))}{...
3
votes
0answers
158 views

Gibbs sampler for local linear trend model

Question: Consider the local linear trend model given by: \begin{align*} y_t = \mu_t + \tau \varepsilon_t \ \cdots \ \text{Observation equation} \\ \mu_{t+1} = \phi \mu_t + \eta_t \ \cdots \ \text{...
3
votes
0answers
294 views

Sampling Stationary Vector Autoregression coefficients while Gibbs Sampling

I have been estimating a Bayesian Vector Autoregression using Gibbs Sampling. When constructing the posterior predictive distribution, I have noticed that when the simulated coefficients from the MCMC ...
2
votes
0answers
39 views

Gibbs updating algorithm (Gibbs steps) for computationally expensive likelihood

I am looking for a good way to update steps in a Gibbs sampler where the likelihood function is computationally expensive. Here is what I tried so far: By default JAGS uses a slice sampler. However, ...
2
votes
0answers
51 views

Gibbs sampler of a generative model

I understand what a Gibbs sampler is and I understand how LDA does classification. But I'm unsure how I can generate a Gibbs sampler for an LDA model and how to meld the two concepts. Let's say I ...
2
votes
0answers
72 views

Intuition on why Gibbs Sampling samples from the posterior distribution

I am new to Gibbs Sampling and I do understand how the algorithm works but I would also like to understand how sampling from the conditional distributions is equivalent to sampling from the joint. ...
2
votes
0answers
39 views

Gibbs sampling for mixture with Dirichlet prior?

I want to sample from the distribution of a mixture distribution. The hierarchical model is $x_i\sim f$, where: $$f(x\mid \theta_1,\dots,\theta_p, w_1,\dots,\omega_p) = \sum_{j=1}w_p\varphi(x\mid\...
2
votes
0answers
68 views

FFBS algorithm for estimating mean log-return parameter in stochastic volatility jump model

I am currently attempting to replicate this model: https://arxiv.org/pdf/1809.01501.pdf in r. My (first) problem is regarding how to sample from conditional posterior for mu, $(μ_{(j)}|Y_n, J_{(j−1)}...
2
votes
1answer
51 views

Sampling from partial posterior

I'm reading a paper where the authors have something like the following as one step in their MCMC: $$ \begin{align} y&=\rho_1 x_1+\rho_2 x_2+\epsilon\\ z&=\beta\tilde{y}+u \end{align} $$ ...
2
votes
0answers
106 views

Can Bayesian Optimization solve this problem?

Suppose ${\bf{x}} = (x_1,\ldots,x_n)$ and $f({\bf{x}})\propto 1_A({\bf{x}}) \prod_{i=1}^n {x_i}^{\alpha_i-1} e^{-\beta_i x_i}$ , i.e. $f$ is proportional to the product of independent gamma ...
2
votes
0answers
564 views

Bayesian prior and posterior computation for a truncated normal

I have to deal with data in a Bayesian framework, ultimately devising a Gibbs sampler for inferring all my distributions parameters. Specifically, suppose I observe some univariate data distributed ...
2
votes
0answers
267 views

Missing data in Gibbs sampling for dynamic linear models

Suppose I have the following DLM: $x_t = \Phi x_{t-1} + w_t$ $y_t = A x_t + v_t$ $x_0 \sim N(\mu_0,\Sigma_0)$ $w_t \sim N(0,Q)$ $v_t \sim N(0,R)$ Let $\Theta = \{\mu_0,\Sigma_0,\Phi,Q,A,R\}$. I ...
2
votes
0answers
444 views

Gibbs sampling in the Hierarchical Dirichlet Process

For an inference problem using a Dirichlet Process prior, one can derive a "basic" Gibbs sampling scheme, where we have a conditional for any parameter $\theta_i$ given the samples $x_i$ and all the ...
2
votes
0answers
32 views

A verifiable and teachable Gibbs example

I am attempting to construct a teachable example of Gibbs sampling that I can also relate to how it might be used on an actual dataset and yet could also be verified analytically by students with ...
2
votes
0answers
1k views

Deriving mean and variance of the posterior distribution

I have a simple linear model: $y_{i}=\mu+e_{i}$ for $i=1,...,n$, where $P(e_{i})=w\mathcal{N}(0,\sigma^2) + (1-w)\mathcal{N}(0,k^2\sigma^2)$ with $w=0.9$, $k=10$ and $\sigma=0.1$. It can be understood ...
2
votes
0answers
152 views

If all components of a hierarchical model have not converged, can we say that any parameters have truly converged?

I'm working with a hierarchical regression model of the following form similar to that presented in Peter D. Hoff's book, A First Course in Bayesian Statistical Methods: $\boldsymbol{Y}_j \sim \text{...
2
votes
0answers
138 views

Estimation of arithmetic Brownian motion volatility with transformed data

I want to estimate the volatility $\sigma$ of a process $(X_t)$ following an arithmetic Brownian motion, that is, for a constant time step $\Delta$, $X_{t+\Delta} = X_t + \sigma B_{\Delta}$ , where $...
2
votes
1answer
324 views

What does it means of Normalization term of Gibbs distribution?

I am studying about Gibbs distribution concept and I am confusing a one term in that concept that is normalization term. According to the Hammersley–Clifford theorem, an random $x$ can equivalently be ...
2
votes
0answers
94 views

Can I use Adaptive MCMC in any setting?

In time series econometrics and finance, most Bayesian authors approximate their models with a Gibbs Sampler, this is especial true for state space models, SV and so forth. The dimensionality of ...
2
votes
0answers
253 views

Gibbs sampling version for estimating Hierarchical Double Dirichlet Process Mixture of Gaussian Processes

I'm trying to implement Gibbs sampling to estimate the parameters of the following non-parametric model: $$\begin{align*} \beta|\gamma & \sim \text{GEM}(\gamma)\\ k_t|\beta & \sim \beta\\ \pi|\...
2
votes
0answers
227 views

Conjugate prior for multivariate with known mean and covariance known to a constant

I have a linear trend model (evolving mean and slope) embedded in a larger state space time series model that I would like to constrain to be a spline. With that assumption, the mean and trend ...
2
votes
0answers
152 views

Understanding a measure of convergence of MCMC simulations

I am trying to better understand better the Gelman/Rubin measure of convergence of MCMCs. The method starts off by defining two quantities: $B$ and $W$. $B$ is said to be the between chain variance (...
2
votes
0answers
186 views

Gibbs sample from AR(1) of exogenous input

I am trying to fit a model where there is a sequence of exogenous "shocks", $X_1, X_2, ..., X_T$, and a AR(1) of these shocks explain $Y_1, Y_2, ..., Y_T$. Specifically, Data (known): $X_1, X_2, ...,...
2
votes
0answers
98 views

Mixture of probits: understanding truncated-based likelihoods

I am trying to implement a mixture model of probits to infer the best decision boundary for every latent subpopulation. When doing Gibbs sampling, we eventually have to compute $P(y^* | w_c)$ where $...
2
votes
0answers
427 views

Practical Implementation of Gibbs Sampling in Latent Dirichlet Allocation

In the collapsed Gibbs sampling version of LDA, the posterior distribution of topic assignments for each word is sampled. From what I have read (e.g. http://people.cs.umass.edu/~wallach/courses/s11/...