Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [gibbs]

The Gibbs sampler is a simple form of Markov Chain Monte Carlo simulation, widely used in Bayesian statistics, based on sampling from full conditional distributions for each variable or group of variables. The name comes from the method being first used on Gibbs random fields modeling of images by Geman and Geman (1984).

8
votes
1answer
708 views

Can I subsample a large dataset at every MCMC iteration?

Problem: I want to perform a Gibbs sampling to infer some posterior over a large dataset. Unfortunatelly, my model is not very simple and thus sampling is too slow. I would consider variational or ...
1
vote
1answer
51 views

Gibbs sampler -transformation of conditional posterior

If my conditional posterior $\pi(\sigma^{-2}|\mathbf{y },\mu)\sim Gamma(a,b)$, how can I get the conditional posterior $\pi(\sigma^{2}|\mathbf{y },\mu)$ with a transformation? The reason I ask is ...
2
votes
1answer
200 views

Gibbs sampling from conditional full posterior distribution

I am reading the paper by Willemsen et al (2015), "A multivariate Bayesian model for embryonic growth", Statistics in Medicine, 34:8, 1351–1365 where they define the posterior distribution as, \begin{...
6
votes
2answers
5k views

Gibbs sampling how to sample from the conditional probability? Bayesian model

I want to learn Gibbs sampling for a Bayesian model. How can I sample the variable from the conditional distribution? In this example, arrow means dependent; for example, ...
0
votes
1answer
152 views

use inverse Wishart for variance in MCMC

When you have a posterior that looks like as one step in Gibber Sampler $P(\xi | \Sigma_\xi, \theta) ∝ exp\{-1/2 \xi\Sigma_\xi^{-1}\xi\}P(data | \xi, \theta)$ Do you always assume inverse Wishart ...
1
vote
1answer
340 views

which R package is good for gibbs sampler when the likelihood function is complex

I see a lot of examples using MCMC to solve for posterior distribution when the likelihood is simply one of linear regression. What if the likelihood is an ugly, complex function. Which R package can ...
5
votes
1answer
97 views

Estimation of Bayesian Models

I'm trying to get into Bayesian model estimation (I'm interested in posterior parameter distributions). I could get away with Metropolis-Hastings and Gibbs Sampling for models with few parameters (<...
2
votes
0answers
300 views

Gibbs sampling in the Hierarchical Dirichlet Process

For an inference problem using a Dirichlet Process prior, one can derive a "basic" Gibbs sampling scheme, where we have a conditional for any parameter $\theta_i$ given the samples $x_i$ and all the ...
1
vote
0answers
110 views

Gibbs Sampling / Monte Carlo with Weights

Consider pairs of data and their population weights $(y_i, \omega_i), i = 1, 2, \dots$ alongside some hierarchical structure, $$y_i\leftarrow\theta_{i(j)} \leftarrow \gamma$$ where $i(j)$ is perhaps ...
1
vote
0answers
174 views

Get different results with different sampling order in Gibbs sampling: what could be wrong?

In sampling a complex spatio-temporal model by Gibbs sampling, I found if I change the order of sampling (for example, to sample $P(\theta_1,\theta_2|D)$, in one try, I sample $\theta_1\sim P(\theta_1|...
1
vote
2answers
1k views

Unsupervised Bayesian naive Bayes

I'm reading a paper Gibbs sampling for the uninitiated. In this paper, the authors try to use Gibbs sampling for a bayesian naive bayes model. They formalize the model as a graphical model in page 8. ...
0
votes
1answer
134 views

Eq. 45-46 in Gibbs Sampling for the Uninitiated

I am trying to figure out how Eq. 45 simplifies to Eq. 46 in the paper - "Gibbs Sampling for the Uninitiated" by Resnik and Hardisty.www.cs.umd.edu/~hardisty/papers/gsfu.pdf (page 15) Eq. 45 $$ \...
2
votes
1answer
140 views

Is Coordinate Ascent algorithm related to Gibbs Sampling in some way?

I wonder if only me feel there are certain connections between them, I googled it for a long time, but found no where mentioned these two method. But to me, they indeed looked so related, Could anyone ...
2
votes
2answers
331 views

Gibbs Sampler, Bivariate Normal, Subchain

In Example 7.1 of "Introducing Monte Carlo Methods with R", the authors write $(X,Y)\sim N\Bigg((0,0),\begin{pmatrix}1 &\rho \\ \rho & 1\end{pmatrix}\Bigg)$ Then, Given $x_t$, $Y_{t+1}\mid ...
0
votes
2answers
245 views

stationarity of vector autoregression and Gibbs sampling

I'm estimating a vector autoregression (VAR) using Gibbs sampling. At each iteration, I'd like to check the coefficients to ensure the VAR is stationary. An older, related question has been posted ...
1
vote
1answer
82 views

Gibbs Sampler Consistency

Simple question that I haven't found easily online: why are the estimates obtained from a Gibbs sampler consistent (converge to the true probabilities)?
1
vote
0answers
26 views

Sampling from a truncated random effects distribution

How would one sample observations $T_{ij} = U_i + \varepsilon_{ij}$, where the distribution of $U_i, \varepsilon_{ij}$ are known and mutually independent, condition on the fact that $L_{ij} \le T_{ij} ...
2
votes
1answer
27 views

Gibbs Sampler while fixing one parameter

I came across a problem for Gibbs sampler. Suppose I want to draw samples from $f(x,y,z)$, Can I use the following scheme to draw samples? Step 1, draw $f(x^{2t-1},y^t|z^{t-1})$. Step 2, draw $ f(...
2
votes
2answers
2k views

Metropolis-Hastings Algorithm within Gibbs Sampling

I have this $f$ function below. $$ f(x_1,x_2)\propto \left(\dfrac{x_1}{x_2}\right)\left(\dfrac{\alpha}{x_2}\right)^{x_1-1}exp\left\{-\left(\dfrac{\alpha}{x_2}\right)^{x_1} \right\}I_{R^+}(x) $$ where ...
3
votes
1answer
185 views

What is a hierarchical model that can estimated via the Metropolis-Hastings Algorithm but not the Gibbs Sampler?

My understanding of the differences between MH and Gibbs Samplers is that a Gibbs Sampler is usually used when the full conditionals are present to us. In other words, it is a known distribution, so ...
3
votes
0answers
487 views

Bayesian estimation using Gibbs sampling for financial models

I am trying to do Gibbs sampling, from this paper. This is a CIR financial model, I want to do Gibbs on its parameters: $$y(t+{\Delta}^{+})=y(t)+(\alpha-\beta y(t)){\Delta}^{+}+\sigma \sqrt{y(t)}{\...
1
vote
1answer
340 views

Sampling from the joint distribution p(x,y) when y = f(x)

Suppose I want to sample from the joint distribution $p(X, Y)$, where $X$ is a random variable and $Y = f(X)$ where $f$ is a known function of $X$. However, sampling from $p(X,Y)$ directly is hard. ...
2
votes
1answer
361 views

Gibbs sampling on the use of gamma distribution

I am trying to reproduce a Gibbs sampling but some points are unclear to me. Let's assume a linear model of the following form $$y=x\beta + \epsilon$$ The prior distribution for the beta ...
2
votes
2answers
2k views

How does Gibbs sampling produce values for a variable using the univariate conditional probability?

I have a question about Gibbs sampling for generating samples. The Gibbs sampling algorithm is often stated. $x^0 = (x_1^0, x_2^0, \ldots, x_n^0)$ //initialize random values for $t=1$ in $T$ //...
3
votes
0answers
688 views

Bayesian estimation of Dynamic Regression with AR(1) parameters

I would like to draw (Bayesian) inference in a dynamic linear regression with regression parameters following independent AR(1) processes $\beta_{t,i} = \mu_i+\beta_{t-1,i}+w_{t,i}$. However, I ...
1
vote
2answers
156 views

throwing away all Gibbs samples after approximation

This is more of a theory question, consider: $$P(w_1|D)=\int P(w_1|S)P(S|D)d(S)$$ which we approximate via Gibbs sampling $S$ (assume the initial state of the Gibbs sampler is denoted by $M_0$), ...
3
votes
1answer
166 views

Calculating conditional probability

Let's consider I have the pair of distributions: \begin{align} X|t &\sim Binomial(n,t) \\ t &\sim Beta(a,b)$ \end{align} Here $n,a,b$ are known. I need to construct conditional probability ...
2
votes
0answers
105 views

Estimation of arithmetic Brownian motion volatility with transformed data

I want to estimate the volatility $\sigma$ of a process $(X_t)$ following an arithmetic Brownian motion, that is, for a constant time step $\Delta$, $X_{t+\Delta} = X_t + \sigma B_{\Delta}$ , where $...
4
votes
1answer
189 views

handling metropolis hastings rejection during a Gibbs sweep

Suppose I have a MCMC involving a 2 step Gibbs sampler. The first part uses metropolis hastings to find the next parameter value. If during one sweep, the result for the first part is a rejections, ...
2
votes
0answers
26 views

A verifiable and teachable Gibbs example

I am attempting to construct a teachable example of Gibbs sampling that I can also relate to how it might be used on an actual dataset and yet could also be verified analytically by students with ...
1
vote
0answers
56 views

Methods to solve a compound distribution MLE I can't write in closed form

What are some methods of solving a MLE parameter estimate where the likelihood function can't be written in closed form? I have a compound distribution that I don't have a closed form for (no ...
1
vote
1answer
544 views

How to do MC integration from gibbs sampling of posterior

I'm a beginner of MCMC. Two questions confused me. If I know the posterior distribution, and from the Gibbs sampling, I got the sampled parameter, so How to draw the histogram with y axis as ...
1
vote
1answer
476 views

Generating Sample Path of HMM via Gibbs Sampling

I have a question regarding section 7.1.1 here. I have two questions to ask you. What does the following sentence mean? we shall be drawing values for $C_T,C_{T-1},\cdots,C_1$ in order. How the ...
1
vote
0answers
47 views

Gibbs sampling confusion

I'm just wondering if i'm doing this process correctly i'm a little confused as to the answers i'm getting: I have $P(x=0,y=0) = P(x=1,y=1) = 0.5$ and $P(x=0,y=1) = P(x=1,y=0) = 0$ I calculated $P(...
4
votes
0answers
587 views

Gibbs sampling for spike and slab priors

In Spike and slab variable selection (equation 4) there is a model setup of the form $\beta_k | \lambda_k, \tau_k \sim \text{Normal} (0, \lambda_k \tau_k^2)$ $\lambda_k | \nu_0, w \sim (1-w)\delta_{\...
3
votes
0answers
287 views

When Gibbs Sampling is fast/slow to converge?

Are there any heuristic/theories showing that on what kind of Bayesian models the convergence of Gibbs sampling is fast/slow? For example, from my limited experience, I feel (may be wrong) when a ...
1
vote
1answer
181 views

Conditional posterior probability density (steps)

I'm trying to understand how to condition a probabilistic posterior distribution. Consider the following probability density: $$ p(\alpha, \beta | y) = \prod_{i=1}^n (\alpha+\beta t_i)^{y_i}e^{-(\...
3
votes
0answers
564 views

Criteria in determining “step size” of Metropolis-hasting algorithms

I am training a complex Bayesian model using Gibbs sampling and Metropolis-Hasting algorithm. Most of the parameters are directly sampled by using conjugate priors except for 3 params which are ...
2
votes
0answers
125 views

Seeking help in Bayesian Mixed Effects Model

I am implementing a Bayesian Mixed Effects model in my research problem. The model is written as, $y_i = X_i(\alpha + \beta_i) + \epsilon_i$, where $i = 1, 2, \ldots, m$ is the index of response, $j = ...
2
votes
1answer
193 views

Does Gibbs sampling MCMC place limitations on the posterior?

I'm starting to learn about Gibbs sampling, having so far only worked with Metropolis-Hastings MCMC, and there's something I haven't grasped yet about the way Gibbs sampling works. To frame the ...
6
votes
1answer
186 views

Dirichlet process mixture MCMC

I'm reading Markov Chain Sampling Methods for Dirichlet Process Mixture Models by Radford M. Neal. Equation (3.6) states that $$ \text{If } c=c_{j} \text{ for some } j\neq i: P\left(c_{i}=c\;|\;c_{-i}...
1
vote
1answer
242 views

Expected value of an estimate done using Gibbs sampling

This is related to this question. I am concerned about the expected value of a function estimated using sampling. If the samples were obtained by a sampling method such as Gibbs sampling, and if I ...
4
votes
1answer
470 views

Reversibility in MCMC

I am reading Geyer's lecture notes on MCMC. A condensed version of these notes constitutes Chapter 1 of the Handbook of Markov Chain Monte Carlo (ed. Brooks et al., 2011). Geyer notes that the ...
0
votes
1answer
130 views

How does this Sampler work for the Concentration parameter of Dirichlet Process?

I am puzzled by how this Gibbs sampler on section 6 of Escobar & West (1995) works. To put it in simple words, the aim is to sample $\alpha$. The defined terms are: $$\eta\sim \texttt{Beta}(a,b)$$ ...
5
votes
0answers
240 views

Bayesian estimates for Deming regression coinciding with least-squares estimates

Consider the following Deming model with independent replicates : $$x_{i,j} \mid \theta_{i} \sim {\cal N}(\theta_{i}, \gamma_X^2), \quad y_{i,j} \mid \theta_{i} \sim {\cal N}(\alpha+\beta\theta_{i}, \...
0
votes
2answers
116 views

Why are all of the estimated coefficients in a linear regression model not significantly different from 0?

I am trying to estimate a linear regression model (in the context of econometrics) using Bayesian approach (Gibbs sampler). The choice of the explanatory variables and model specification can be ...
1
vote
1answer
109 views

How to Gibbs sample proportional to a probability

I am reading this tutorial on Hierarchical Chinese Restaurant Process. On pdf page 141 (slide title: MCMC Problem Specification for N-grams) it says: $$F(s_{1,k})=\frac{\alpha^{S'_1+s_{1,k}}}{(\alpha)...
2
votes
1answer
345 views

Gibbs sampling with constraints

I am reading tutorials on Gibbs sampling for partition sampling in Dirichlet Process (Chinese Restaurant Process), and have been struggling to understand the terminology used in the tutorials. To ...
8
votes
2answers
2k views

Rao-Blackwellization of Gibbs Sampler

I am currently estimating a stochastic volatility model with Markov Chain Monte Carlo methods. Thereby, I am implementing Gibbs and Metropolis sampling methods.Assuming I take the mean of the ...
7
votes
2answers
7k views

Generating samples from Gibbs sampling

I am quite new to sampling. I am doing Gibbs sampling for a Bayesian network. I am aware about the algorithm for the Gibbs sampling but there's one thing I am not able to understand. For example let'...