Questions tagged [metropolis-hastings]

A special type of Markov Chain Monte Carlo (MCMC) algorithm used to simulate from complex probability distributions. It is validated by Markov chain theory and offers a wide range of possible implementations.

Filter by
Sorted by
Tagged with
0
votes
0answers
15 views

Help to Proof of Gibbs sampler acceptance rate

i was wondering if someone could explain how the acceptance rate in the Gibbs Sampling works. In the literature it says that in Gibbs case, the acceptance is always 1 because part of the "...
0
votes
0answers
39 views

Bayesian analysis example with convergence under Gibbs but not Metropolis-Hastings

Having a conceptual understanding of algorithms such as Metropolis-Hastings, Gibbs and Hamiltonian Monte Carlo can provide ideas of remediation to apply when models do not converge. This question ...
2
votes
2answers
156 views

Metropolis Ergodicity

I have encountered one last problem with regarding to the Metropolis-Hastings algorithm. I know that ergodicity is needed in the algorithm to imply convergence to a unique stationary distribution. But ...
12
votes
2answers
550 views

In Bayesian models, can you use Uniform(-inf, inf) as a prior?

In Bayesian models, can you use Uniform(-inf, inf) as a prior? I ask because in an class, we looked at MH MCMC sampler, and showed that to sample from a distribution, we need not explicitly solve for ...
0
votes
0answers
26 views

Metropolis Hastings on hierarchical bayes update question:

[I have this somewhat complicated hierarchical bayesian model]1 Here the $y$ on $\theta$ are Poisson, $\theta$ are deterministically generated from the $att, def$ (and $home$). Then the last ...
0
votes
0answers
11 views

Optimal way to calculate the sample variance of MCMC data?

I assume that the calculation of the average and variance of a sample generated by a Metropolis-Hastings MCMC does not depend on the existence of correlation. Recently, I was told that in the presence ...
3
votes
1answer
525 views

MCMC acceptance rate decreases when proposal variance gets smaller

I am drawing a sample Y of size n from a p-dimensional Normal ($\mu, \Sigma$). Typically, p is 5. I have $\bar{Y}$ and $V = YY'$, the sum of squares. Now I want to draw samples from this $\bar{Y}$, ...
1
vote
1answer
252 views

What does it mean to have assymetric proposal distributions when coding the Metropolis-Hastings Algorithm?

The Metropolis-Hastings algorithm is a generalization of the older Metropolis algorithm. As part of these algorithms, they compute a ratio called the Metropolis ratio: $$ r = \frac{P(x')}{P(x)}\frac{...
5
votes
2answers
1k views

What is the relationship between Metropolis Hastings and Simulated Annealing?

Context and Problem In the Wikipedia page for Simulated Annealing they state The simulation can be performed either by a solution of kinetic equations for density functions[2][3] or by using the ...
0
votes
0answers
27 views

Bayesian multivariate regression with common coefficients

In a hierarchical model I'm working on, I have $K$ different $N\times P$ predictor matrices, each denoted $X_k$ and $K$ length $N$ outcome vectors each denoted $y_k$. Essentially, I have a ...
7
votes
2answers
268 views

Using all Metropolis-Hastings proposals to estimate an integral

Suppose we run the Metropolis-Hastings with target distribution $\mu$ to compute the integral $\int f\:{\rm d}\mu$. Usually, we use the estimator $$A_n:=\frac1n\sum_{i=0}^{n-1}f(X_i).$$ However, ...
0
votes
1answer
29 views

Dynamically adjusting parameters of Markov chain

I am using a Metropolis algorithm to generate samples from a complicated (high-dimensional) probability distribution. As is common, the proposed updates depend on some "step size" parameter $...
45
votes
1answer
18k views

What is the difference between Metropolis-Hastings, Gibbs, Importance, and Rejection sampling?

I have been trying to learn MCMC methods and have come across Metropolis-Hastings, Gibbs, Importance, and Rejection sampling. While some of these differences are obvious, i.e., how Gibbs is a special ...
1
vote
0answers
31 views

Bias in unbiased pseudo-marginal estimation?

In the Pseudo-marginal Metropolis-Hastings algorithm exact sampling of a posterior distribution is performed when using an unbiased estimate of the marginal likelihood. However, I am having problems ...
18
votes
1answer
3k views

Stan $\hat{R}$ versus Gelman-Rubin $\hat{R}$ definition

I was going through the Stan documentation which can be downloaded from here. I was particularly interested in their implementation of the Gelman-Rubin diagnostic. The original paper Gelman & ...
1
vote
1answer
70 views

How to understand the scaling in Metropolis Hastings MCMC

We know the Metropolis Hastings (MH) in MCMC: target distribution: $\pi(x)$ proposal distribution: $p(y|x)$ acceptance: $\alpha(x,y) = \min \Big(1, \dfrac{\pi(y)p(y|x)}{\pi(x)p(x|y)}\Big)$ Here are ...
0
votes
0answers
20 views

Measure to capture within-chain fluctuations in MCMC?

I am using two kinds of updates for a particular parameter in MCMC estimation of my model. First update gives the following trace plot: Second update gives the following trace plot: Note that the ...
3
votes
2answers
750 views

How to use the MCMC method for multivariate distributions?

I wish to find the posterior of a joint distribution of 4 parameters whose prior and likelihoods are known, but I do not understand how to accept and reject samples, in any other case other than ...
3
votes
0answers
28 views

Metropolis-Hastings Reversibility: What is the measure used in the definition of reversibility?

Let $\pi$ be a target probability distribution on a measurable space $(E, \mathcal{E})$. MCMC obtains dependent samples from $\pi$ by using a Markov Chain with transition kernel $\mathrm{K}:E\times \...
1
vote
1answer
79 views

What is the interpretation of this modified Metropolis algorithm?

Modified Metropolis-Hastings Consider a model with parameters $\theta = (\alpha, \gamma)$ and consider a modified Metropolis-Hastings algorithm which can be summarized (with brevity) as follows. ...
3
votes
2answers
270 views

Is Coordinate Ascent algorithm related to Gibbs Sampling in some way?

I wonder if only me feel there are certain connections between them, I googled it for a long time, but found no where mentioned these two method. But to me, they indeed looked so related, Could anyone ...
3
votes
1answer
97 views

Does the MCMC algorithm use the likelihood function to move to a new proposal value?

I'm trying to understand how MCMC and related algorithms work for Bayesian inference. In this paper the authors use an example of a normal distribution in explaining MCMC. They state, "If the ...
0
votes
0answers
63 views

Metropolis Algorithm for a high dimensional Bimodal distribution

I am using metropolis mcmc for an $n=8$ dimensional system on an (n-1)-sphere. I was considering the 2d case, as it can be visualized. For this case, the probability density,up to a normalization, is \...
4
votes
1answer
61 views

Mean acceptance rate for Metropolis-Hastings algorithm

My question relates to the result stated on page 4 of: http://stat.columbia.edu/~gelman/research/published/baystat5.pdf which claims that the mean acceptance probability when performing the Metropolis-...
3
votes
1answer
95 views

Rao–Blackwellization of Metropolis–Hastings

I am trying to achieve a Rao–Blackwellization of Metropolis–Hastings algorithm. In the paper by Robert et al. 2018, the following is given. \begin{align} ℑ=&\frac{1}{T}\sum_{t=1}^Th(\theta^{(t)})=\...
0
votes
1answer
47 views

Question about MCMC independent proposals

I've a question re MCMC proposals, I was hoping you could help me with that. I need to implement for work an Independent Metropolis-Hastings algorithm to sample from a 10-dimensional posterior. I am ...
2
votes
1answer
91 views

When is metropolis Hastings algorithm useful

I need to do a data analysis project and am considering the Metropolis Hastings algorithm to estimate the parameters of a logistic regression model. I would draw from the complete data log likelihood ...
1
vote
1answer
41 views

Is it possible to improve Markov Chain Monte Carlo performance by decomposing a Binomial Likelihood?

Suppose that we have sampled $y_{1},y_{2},...,y_{n}$ from a Binomial distribution $Bin(N,p)$. Also, let's assume that $p$ is known and our goal is to infer the unknown parameter $N$, with the use of ...
1
vote
0answers
41 views

MCMC algorithm for Hierarchical Bayes model with variable number of mixture components

I am trying to develop an MCMC algorithm for clustering $n$ data-points $y_{1},y_{2},\dots,y_{n}$ using a Gaussian mixture model, but with a prior defined on the number of components K. The ...
2
votes
0answers
24 views

Drift and Minorization for Metropolis Hastings algorithm

Can some one point me to articles or literature with an example of the drift and minorization condition proof. At the moment i have come across the Gibbs Sampler and the Random Walk Metropolis ...
1
vote
0answers
49 views

Best Proposal Distribution for asymmetric target in Metropolis Hastings (Exponential Target)

Suppose I know my target distribution is asymmetric, for instance, suppose my target distribution is an exponential $$ p(x) = e^{-x} \qquad\qquad x\in [0, +\infty) $$ Is there any theory regarding ...
0
votes
0answers
46 views

High Acceptance rate with Uniform Proposal Distribution

I'm using a metropolis algorithm to model a physical system. There are N sets of 8 real parameters, each set of 8 real numbers is normalized. The target is a Boltzmann distribution with the energy for ...
0
votes
1answer
55 views

Stratification of Random Walk Jumps in Metropolis-Hastings

I aim to achieve variance reduction in Random Walk Metropolis Hastings algorithm by introducing stratification to the random walk jumps. What I have tried is to make use of Latin Hypercube Sampling in ...
1
vote
0answers
15 views

Gibbs sampling step for variables that have a complex offline prior in an MCMC hybrid

I have a question about how to use an offline function as a prior when performing a Gibbs/hybrid analysis. Let's say I have data $y$ and some parameters which I'll simplify to $\theta_1, \theta_2$. ...
0
votes
0answers
107 views

How to sample from a joint probability distribution of two variables? [duplicate]

I want to randomly sample from a joint probability distribution characterized by two variables x and y. So, basically, I have the information of the joint PDF which has the dimension 100x50. As per my ...
-1
votes
1answer
288 views

Metropolis Hastings with Gamma Proposal Density

I am trying to use Metropolis Hastings to sample from a shifted gamma distribution. Since it is shifted, it has a domain of $(n, \infty)$. I tried using a Gaussian proposal density and ran into the ...
2
votes
1answer
118 views

Monte Carlo Methods: [closed]

Can someone explain to me the following statement from “Introducing Monte Carlo methods with R!” By Robert Christian. “If the exploration mechanism has enough energy to reach as far as the boundaries ...
1
vote
1answer
67 views

Random Walk Metropolis: acceptance probability with truncated normal proposal

I want to draw from my target density $p(\theta)$ using Random Walk Metropolis. $\theta$ has domain $[2, +\infty)$, and I am using as proposal a truncated normal, namely: $$q(\theta_t') \sim N(\theta_{...
1
vote
1answer
125 views

Application of Metropolis Hastings

I am trying to implement the Metropolis Hastings algorithm for Bayesian analysis. In this case, the parameter of interest is the scale parameter for a Weibull distribution. The context is for ...
0
votes
1answer
76 views

Computing the acceptance rate (empirically) of samples from the Metropolis algorithm, where samples are "thinned"

I have a number of queries about computing the acceptance rate of samples generated from the Metropolis (symmetric random walk) algorithm empirically, that is, in the presence of burning-in and ...
0
votes
0answers
55 views

Thinning and burn-in in Metropolis-Hastings algorithm

I have written a Metropolis-Hastings algorithm manually in Julia language for a customized distribution, and i want now to know how can i perform the thinning and the burn-in to increase the ...
0
votes
1answer
50 views

How do we define the kernel to calculate the acceptance ratio for Metropolis-Hastings Markov Chain Monte Carlo?

I am having a lot of difficulty understanding how to apply the algorithm to a real scenario. The part that confuses me is that we are looking for a target distribution (the real distribution of our ...
0
votes
1answer
102 views

Metropolis-Hastings with non-centered Proposal

I am trying to draw samples from the Laplace distribution $\pi^* = \text{exp}(-|\theta|)$, using Metropolis Hastings algorithm with a noncentered proposal, meaning that regular Metropolis wont work.. ...
1
vote
0answers
188 views

what is the optimal step size for metropolis-hastings algorithm to have independent state

In the PRML chapter 11, The Metropolis-Hasting algorithm, For a sampler with Gaussian distribution as proposal distribution. The original distribution is correlated multivariate Gaussian distribution, ...
1
vote
0answers
32 views

Metropolis-Hastings algorithm for a completely specified distribution

Consider a random variable $X\sim f(x)$, such that $$ f(x)=\frac{1}{c}\times K(x)\propto K(x), $$ where c: normalizing constant, K(x): the kernel of the distribution (ie the part which involves $x$). $...
0
votes
3answers
170 views

Real-life example in which Markov chain Monte Carlo is desirable? [duplicate]

A typical introduction to the Metropolis--Hastings algorithm, and hence to Markov chain Monte Carlo techniques in general, starts with the following assumptions on some probability distribution $P(x)$ ...
1
vote
0answers
31 views

Metropolis Hastings for BART: Calculation of Tree Prior and Transition Kernel

I am trying to understand the details of BART (Bayesian Additive Regression Trees). In particular, I would like to know how the Metropolis Hastings acceptance probability is calculated for BART. My ...
2
votes
0answers
39 views

Gibbs updating algorithm (Gibbs steps) for computationally expensive likelihood

I am looking for a good way to update steps in a Gibbs sampler where the likelihood function is computationally expensive. Here is what I tried so far: By default JAGS uses a slice sampler. However, ...
0
votes
0answers
87 views

Metropolis Hastings Stuck in Local maxima

I've been running the metropolis hastings algorithm to infer some parameters. After running multiple chains, there are typically two places the chains get stuck in, one of which has a higher ...
0
votes
1answer
198 views

Metropolis Hastings algorithm for joint posterior of probability of heads for 2 coins

I am trying to implement a simple metropolis hastings algorithm to simulate the joint posterior of the probability of flipping heads for 2 coins, $\theta_1,\theta_2$. I am following the problem ...

1
2 3 4 5
9