Questions tagged [mcmc-acceptance-rate]

The acceptance rate (acceptance ratio, acceptance fraction) for a Markov Chain Monte Carlo sampler indicates the fraction of accepted over proposed moves.

Filter by
Sorted by
Tagged with
3 votes
1 answer
48 views

Derivation of acceptance probability from Linero, Yang (2018)

I am wondering how this paper Bayesian Regression Tree Ensembles that Adapt to Smoothness and Sparsity by Linero & Yang (2018) derived the acceptance probability for $\sigma$. The authors give $\...
xiaoshuxiaowei's user avatar
0 votes
1 answer
72 views

Jacobian and proposal ratio of Birth/death step in RJMCMC of Gaussian mixture model

I am asking questions regarding RJMCMC several times in this site. Some of my questions are answered and some are unanswered. It didn't clarify all of my unclear points but I am glad that I have ...
Kyungmin's user avatar
1 vote
1 answer
245 views

Is Metropolis-Hastings ever more efficient than rejection sampling in 2 dimensions?

I know that Metropolis-Hastings is an MCMC (Markov Chain Monte Carlo) method that is very useful in higher dimensions. The advantages it has over something like simple rejection sampling are that ...
Aditya S's user avatar
0 votes
0 answers
133 views

Random scan Gibbs sampling as special case of Metropolis-Hastings

I am reading Blitzstein's Introduction to Probability and come across with the following proof that I don't really understand: Theorem: The random scan Gibbs sampler is a special case of the ...
Borcherdsbb's user avatar
0 votes
0 answers
12 views

Is it possible to increase the Hastings ratio by combining and mixing elementary kernels?

Let's say I am working with a state $X$ split into three parts $U$, $V$, and $W$. I can efficiently sample from $W|U,V$, $U|V$, and $V|U$. My initial intuition was to do a variable-at-a-time ...
J. Zeitouni's user avatar
0 votes
1 answer
116 views

Computational aspect of the Metropolis-Hastings algorithm

One of the examples online is about how to write the Metropolis-Hastings algorithm from scratch. This tutorial uses a linear regression model as an example. Estimate three parameters: an intercept, a ...
Keith Lau's user avatar
  • 115
0 votes
0 answers
67 views

Metropolis Hastings Algorithms: How to measure the performance of algorithms? (Multidimensional)

I am working on a project and I am trying to measure the performance and compare two MCMC algorithms. The one is Random-Walk MH and the second one is PCN. I thought of maybe comparing the mean ...
makala's user avatar
  • 11
2 votes
2 answers
375 views

MCMC - How to derive the acceptance ratio from Markov chain detailed balance?

Please explain how the acceptance ratio: $$\rm A(x\to y) =\min\left(1,\frac{P(y)q(y\to x) }{P(x) q(x\to y) }\right)$$ is derived from the detailed balance: $$\rm \frac{A(x\to y) }{A(y\to x)}=\frac{P(y)...
mon's user avatar
  • 1,468
2 votes
0 answers
105 views

The guidelines for choosing different MCMC algorithms [closed]

MCMC has several types of algorithms: Metropolis-Hastings, Gibbs, Adaptive MH, Hamiltonian Monte Carlo. What are their respective pro/cons, and how to choose them in the Bayesian analysis?
user3269's user avatar
  • 5,162
3 votes
1 answer
441 views

Why do we sample from the uniform distribution in Metropolis-Hastings for acceptance?

For each iteration of the MH, sample $x'=q(x|x')$, then the acceptance probability is computed:$$A=\min(1,a)$$ where $$ \alpha=\frac{p(x')q(x|x')}{p(x)q(x'|x)} $$ Now, I've seen that the algorithm ...
wd violet's user avatar
  • 777
0 votes
1 answer
167 views

How can I find the acceptance probability for a joint Metropolis-Hastings proposal?

Suppose that I want to generate a proposal $(x^*,y^*,z^*)$ with the following: $$z^*\sim p(z|\alpha,\beta)$$ $$x^*\sim p(x|z^*,\boldsymbol{\gamma}_x)$$ $$y^*\sim p(y|z^*,\boldsymbol{\gamma}_y),$$ ...
Oski's user avatar
  • 5
2 votes
1 answer
359 views

How to understand the scaling in Metropolis Hastings MCMC

We know the Metropolis Hastings (MH) in MCMC: target distribution: $\pi(x)$ proposal distribution: $p(y|x)$ acceptance: $\alpha(x,y) = \min \Big(1, \dfrac{\pi(y)p(y|x)}{\pi(x)p(x|y)}\Big)$ Here are ...
user6703592's user avatar
  • 1,325
2 votes
0 answers
46 views

How to apply MCMC to bayes when likelihood is not easy to compute

Let $z$ be observations and $w$ be the parameter that we want to infer. Assuming that we know the prior $p(x)$, by using Bayes law, we have $p(x|z) = p(z|x)p(x)/p(z)$ where $Z$ is the marginal ...
orematasaburo's user avatar
4 votes
1 answer
93 views

Mean acceptance rate for Metropolis-Hastings algorithm

My question relates to the result stated on page 4 of: http://stat.columbia.edu/~gelman/research/published/baystat5.pdf which claims that the mean acceptance probability when performing the Metropolis-...
Matt Barrett's user avatar
1 vote
1 answer
1k views

Metropolis Hastings for Poisson Distribution

Studying Bayesian Inference and Markov chain Monte Carlo (MCMC) algorithms, I am facing a self study question on a MCMC approach to a Poisson distribution with parameter $\lambda$. Using R, my code is:...
Arduin's user avatar
  • 139
1 vote
0 answers
469 views

How can I tune my Random Walk Metropolis Hastings algorithm on the fly?

I just have a very general question. I've recently started to use Random Walk Metropolis-Hastings (RWMH) to sample from a distribution in order to calculate integrals. But I've noticed that the ...
AlphaBetaGamma96's user avatar
0 votes
1 answer
269 views

How do we define the kernel to calculate the acceptance ratio for Metropolis-Hastings Markov Chain Monte Carlo?

I am having a lot of difficulty understanding how to apply the algorithm to a real scenario. The part that confuses me is that we are looking for a target distribution (the real distribution of our ...
Radu's user avatar
  • 1
1 vote
0 answers
555 views

what is the optimal step size for metropolis-hastings algorithm to have independent state

In the PRML chapter 11, The Metropolis-Hasting algorithm, For a sampler with Gaussian distribution as proposal distribution. The original distribution is correlated multivariate Gaussian distribution, ...
Kelvin_neverKnow's user avatar
1 vote
0 answers
79 views

Metropolis-Hastings algorithm for a completely specified distribution

Consider a random variable $X\sim f(x)$, such that $$ f(x)=\frac{1}{c}\times K(x)\propto K(x), $$ where c: normalizing constant, K(x): the kernel of the distribution (ie the part which involves $x$). $...
Ludwig's user avatar
  • 63
1 vote
0 answers
386 views

Jacobian term for Metropolis Hastings algorithm?

Suppose an acceptance ratio of the MH sampler without parameter transformation is like this: \begin{gather} r=\frac{f( \theta ',\gamma '|y,m) q( \theta ',\gamma '|\theta ,\gamma )}{f( \theta ,\gamma |...
Keith Lau's user avatar
  • 115
1 vote
0 answers
132 views

Adaptive MCMC with low acceptance rate

I'm running an MCMC scheme defined in A tutorial on adaptive MCMC. Christophe Andrieu·Johannes Thoms. Stat Comput (2008) 18: 343–373DOI 10.1007/s11222-008-9110-y. They say the optimal scaling factor ...
user112495's user avatar
1 vote
2 answers
278 views

Simulated annealing acceptance probability puzzle

My understanding of simulated annealing (SA) is that at any iteration $t$, a new sample $Y_t$ is generated, which, if the objective function $E$ is improved, i.e., $E(Y_t)<E(X_{t-1})$, then $Y_t$ ...
user138668's user avatar
1 vote
1 answer
149 views

How do we need to define the acceptance ratio of the Metropolis-Hastings algorithm for a vanishing proposal density?

I've seen that different authors define the acceptance probability $\alpha$ of the Metropolis-Hastings algorithm with target distribution density $p$ and proposal kernel density $q$ differently; some ...
0xbadf00d's user avatar
  • 283
0 votes
1 answer
269 views

How do I account for numerical overflow with Adaptive MCMC?

EDIT: I tested Forgottenscience's solution below and it works; however, note that I found the working acceptance criterion to be that if $\log\alpha \geq \log u$, the point is accepted, where $u\sim\...
SandwichTheorem's user avatar
3 votes
1 answer
898 views

Do I need to evaluate acceptance rates in Metropolis within Gibbs algorithm?

Consider the Gibbs sampler Sample $\theta' \sim p(\theta|\tau, D)$ Sample $\tau' \sim p(\tau|\theta', D)$ where $\theta,\tau$ parameters of the data $D$. Now assume that we can only sample from $p(\...
tomka's user avatar
  • 6,602
1 vote
1 answer
103 views

Can a Metropolis-Hastings estimator converge if the proposal density vanishes whenver the target density vanishes?

I'm running the Metropolis-Hastings algorithm for a target and proposal density $p$ and $q$, respectively, for which it holds $$p(y)=0\Rightarrow q(x,y)=0\;\;\;\text{for all }y.\tag1$$ By definition ...
0xbadf00d's user avatar
  • 283
3 votes
1 answer
106 views

Erroneous expression for Metropolis-Hastings acceptance ratio in a paper

Let $(E,\mathcal E)$ be a measure space; $\rho:E\to[0,\infty)$ be $\mathcal E$-measurable, $p:E^2\to[0,\infty)$ be $\mathcal E^{\otimes2}$-measurable, $$r(x,y):=\left.\begin{cases}\displaystyle\frac{\...
0xbadf00d's user avatar
  • 283
3 votes
2 answers
567 views

Need to understand a statement for Random Walk Metropolis algorithm's proposal distribution?

I was told that the proposal distribution of Random Walk Metropolis needs to be symmetric. But today I was reading a book about Bayesian Analysis which contains the following statement: "The proposal ...
xabzakabecd's user avatar
  • 3,465
3 votes
1 answer
289 views

How to tune MCMC with unwieldy posterior [duplicate]

Let's say I have $n$ observations of a random variable, $X_1, \dotsm, X_n \sim \mathcal{N}(0, \sigma^2)$. I also assume $\sigma^2$ has a Gamma(1,1) prior distribution, $\pi(x) = \exp(-x)$. I'm now ...
CLL's user avatar
  • 229
1 vote
0 answers
216 views

convergence and efficiency of mcmc chains and estimation of covariance matrix

I am doing some bayesian analysis and exploring posterior distribution with mcmc method. I would like some clarification with estimating the covariance matrix. I have a model with 6 parameters. ...
simona's user avatar
  • 119
3 votes
1 answer
274 views

Expression for the mean acceptance rate of the Metropolis-Hastings algorithm

Let $(E,\mathcal E,\lambda)$ be a measure space $p:E\to[0,\infty)$ be $\mathcal E$-measurable with $$c:=\int p\:{\rm d}\lambda\in(0,\infty)$$ and $$\mu:=\underbrace{\frac1cp}_{=:\:\tilde p}\lambda$$ $...
0xbadf00d's user avatar
  • 283
1 vote
0 answers
507 views

MCMC Metropolis-Hastings sampler - Estimation of multiple parameters

First time that I ask a question on this platform! Here I go... I have a dataset with two random variables X1 and X2 and an output Y which comes from a discrete Weibull distribution. I've been trying ...
StevenVDL's user avatar
2 votes
0 answers
153 views

Why volume preservation is important for Metropolis update? [duplicate]

I think my question is naive but I would like to ask why why volume preservation is important for MCMC and specifically Metropolis update.I'm reading the following paper https://arxiv.org/pdf/1206....
G1I2A's user avatar
  • 85
6 votes
1 answer
166 views

How does the celebrated result about the diffusion limit of the Random Walk Metroplis-Hastings algorithm help us to find the optimal scaling

Let $d\in\mathbb N$ with $d>1$ $\ell>0$ $\sigma_d^2:=\frac{\ell^2}{d-1}$ $f\in C^2(\mathbb R)$ be positive with $$\int f(x)\:{\rm d}x=1$$ and $g:=\ln f$ $Q_d$ be a Markov kernel on $(\mathbb R^...
0xbadf00d's user avatar
  • 283
1 vote
1 answer
254 views

Proposal in MCMC lives in bigger space than parameter space. Which transformations should I choose?

I'm using a MCMC algorithm. The proposal is, due to lack of information on my part, a multivariate T-Student distribution, i.e. $\theta \sim \mathcal{MT}(\mu, \Sigma)$. However, some of the components ...
An old man in the sea.'s user avatar
4 votes
0 answers
194 views

Is the MC produced by HMC reversible?

I know that the deterministic dynamics in Hamiltonian Monte Carlo/Hybrid Monte Carlo are reversible and the numerical integrators one uses to approximate them are reversible too. But HMC consists of 2 ...
Lochend's user avatar
  • 51
1 vote
1 answer
145 views

2 versions of Metropolis-Hastings : are they equivalent?

I have seen 2 different versions of Metropolis algorithm. First one : Second one : I don't understand the differences between the 2 versions, especially in the second one where I have to use the ...
user avatar
2 votes
1 answer
3k views

Metropolis Hastings - Acceptance ratio, proposal and lkelihood

From a previous post : First to explain the MH algorithm, it's used to approximate numerically a target distribution, in this case $p(\theta|D)$. At each stage of the algorithm: A value ...
user avatar
5 votes
1 answer
2k views

How worried should I be about low acceptance rate in cold chain (parallel tempering MCMC sampler)

I have a very noisy/multimodal likelihood function for a 6-parameter model. The popular emcee sampler fails miserably (no matter how many chains I use and for how ...
Gabriel's user avatar
  • 4,292