Questions tagged [gibbs]

The Gibbs sampler is a simple form of Markov Chain Monte Carlo simulation, widely used in Bayesian statistics, based on sampling from full conditional distributions for each variable or group of variables. The name comes from the method being first used on Gibbs random fields modeling of images by Geman and Geman (1984).

Filter by
Sorted by
Tagged with
0
votes
0answers
27 views

Gibbs sampling where I can only find the mode of conditionals?

I'm trying to solve a problem with Gibbs Sampling, so I'm trying to do: $$ x_1^1 \sim p(x_1 | x_2^0, x_3^0)\\ x_2^1 \sim p(x_2 | x_1^1, x_3^0)\\ x_3^1 \sim p(x_3 | x_1^1, x_2^1)\\ x_1^2 \sim p(x_1 | ...
0
votes
0answers
21 views

Determine frequent states

I have a distribution from which I can sample (namely, a Boltzmann Machine). Which methods exists to determine frequent states (states with high probability) / the most frequent state (state with the ...
0
votes
0answers
29 views

forward sampling for Bayesian network with continuous variables and equation-based causal relationships

I have a physical system which can be represented by the following Bayesian network. It has the following characteristics 1) The encoded variables are continuous variables 2) The causal ...
0
votes
0answers
86 views

Gibbs sampler for ARIMA AR(1) parameters: division by zero

Suppose the following AR(1) model: $$ y_t = \mu + \phi (y_{t-1} - \mu) + \epsilon_t $$ with $\epsilon_t \sim \mathcal{N}(0,\sigma^2)$. Following issue arises when sampling from $P(\mu_i \;|\; \phi_{...
0
votes
0answers
49 views

bayesian decision making - comparing expected loss

The problem is like this: Suppose that I am considering which country should I invest on, country A and country B, based on their GDP growth rate $\alpha$. There are two possible choices for each ...
0
votes
1answer
180 views

Inferring GMM parameters with Gibbs Sampling

On my book, "Machine Learning A Probabilistic Approach". It's stated that is straightforward to derive a Gibbs sampling algorithm to fit a mixture model, especially if we use conjugate priors. So ...
0
votes
1answer
151 views

How to create a distribution and sample?

Suppose we are given some small set of data on bundles of electrical wires and increasing voltages run through them, and we note how many of the individual wires fail. So for example, a large data ...
0
votes
1answer
118 views

Rewrite conditional formula with three variables using Bayes formula

In equation (5) on page 3 on this paper a conditional probability is rewritten using Bayes' formula. I started using this answer Can I rewrite conditional probability of three variables like this? ...
0
votes
0answers
24 views

Gibbs sampler from from $p(x) = C g(x)$ with $C$ unknown and discrete elements in $X$

By using the Hasting-Metropolis method, is there a way to draw samples from a distribution of this form: $$p(\textbf{x}) = C g(\textbf{x})$$ For $x$ being two dimensional and discrete. The reason that ...
0
votes
1answer
146 views

Eq. 45-46 in Gibbs Sampling for the Uninitiated

I am trying to figure out how Eq. 45 simplifies to Eq. 46 in the paper - "Gibbs Sampling for the Uninitiated" by Resnik and Hardisty.www.cs.umd.edu/~hardisty/papers/gsfu.pdf (page 15) Eq. 45 $$ \...
0
votes
1answer
135 views

How does this Sampler work for the Concentration parameter of Dirichlet Process?

I am puzzled by how this Gibbs sampler on section 6 of Escobar & West (1995) works. To put it in simple words, the aim is to sample $\alpha$. The defined terms are: $$\eta\sim \texttt{Beta}(a,b)$$ ...
0
votes
0answers
56 views

How does GibbsLDA++ ensure that we are sampling from a good posterior?

This is an extending of this question, which asked that whether we should do some estimating to ensure that we are really using a likely topic assignment instead of the one happened with low ...
0
votes
0answers
85 views

Uniform sampling of constrained binary vectors by Gibbs sampling

General statement of the problem: Let $x,y$ be two binary vectors, connected by the following constrains: $$y=f(x),\qquad x=g(y)$$ That is, $x$ determines $y$, and $y$ determines $x$. There are many ...
0
votes
0answers
212 views

Is it possible for Metropolis sampling to converge to the wrong value?

I have simulated data under three parameters of interest, say a, b, c. The prior I put on c was a Gamma, so it only takes positive values. The full conditionals of a and b are known distributions, but ...
0
votes
1answer
83 views

Gibbs sampling: ancillary and sufficient parametrization

After asking a question about Gibbs sampling earlier, I have another one for you. I have not been able to find laymen's background on this, the only referenced use I've found for this is in ...
0
votes
0answers
66 views

How does Shuffled-Complex-Evolution-Metropolis algorithm compare to other adaptive samplers (e.g. NUTS)?

I recently heard of the Shuffled-Complex-Evolution-Metropolis algorithm and am curious how it compares to other adaptive MCMC sampling algorithms. Unfortunately I am still learning about optimizing ...
0
votes
0answers
18 views

What sort of data would be appropriate to analyze under an MCMC method? [on hold]

MCMC methods describe stochastic sampling but I'm not entirely sure the contexts in real datasets one would wish to apply MCMC methods. What kind of data could I gain insight into with MCMC methods?
0
votes
0answers
106 views

Gibbs within Metropolis

Consider a model with two parameters, $\alpha$ and $\beta$. We want to sample these two parameters conditioning on two data points, $d_1$ and $d_2$. Is it possible to use an algorithm like this: 1) ...
0
votes
0answers
211 views

MCMC for Bayesian Inference (Gibbs Sampling) Varying Observed Data

At every step $k$, a Markov chain Monte Carlo algorithm for Bayesian inference with Gibbs sampling draws a parameter of the model to fit, $\beta_i^{(k)}$, from the conditional $Pr\left(\beta_i^{(k)}|\...
0
votes
0answers
189 views

Proper likelihood function in acceptance probability of Gibbs Sampler

I have a question about the acceptance ratio used when implementing a random walk M-H in a gibbs sampler to generate sample paths of an unobservable process. When computing the likelihood of a set of ...
0
votes
1answer
475 views

How can I sample from the conditional distribution?

I am learning Gibbs Sampling, in which there is a step named sampling from conditional distributions. I don't understand: 1. where is the conditional distribution from? From a general case, how can I ...
0
votes
0answers
103 views

How to sample via blocked Gibbs the conjugate normal model?

i am trying to sample the two dimensional parameter $\theta=(\mu,\sigma^2)$, for the Normal model. I have the full conditionals being for the mean: $\pi(\mu_{j}|\ldots) \sim N\left(\frac{\xi_{j}\...
0
votes
0answers
456 views

Test for convergence within Gibbs sampler

I am running a Gibbs sampler for Multivariate Normal times Inverse Wishart posterior distribution with missing data imputation step. I am trying to check if my step of simulating covariance matrices ...