Skip to main content

Questions tagged [mcmc-acceptance-rate]

The acceptance rate (acceptance ratio, acceptance fraction) for a Markov Chain Monte Carlo sampler indicates the fraction of accepted over proposed moves.

Filter by
Sorted by
Tagged with
3 votes
1 answer
48 views

Derivation of acceptance probability from Linero, Yang (2018)

I am wondering how this paper Bayesian Regression Tree Ensembles that Adapt to Smoothness and Sparsity by Linero & Yang (2018) derived the acceptance probability for $\sigma$. The authors give $\...
xiaoshuxiaowei's user avatar
0 votes
1 answer
85 views

Jacobian and proposal ratio of Birth/death step in RJMCMC of Gaussian mixture model

I am asking questions regarding RJMCMC several times in this site. Some of my questions are answered and some are unanswered. It didn't clarify all of my unclear points but I am glad that I have ...
Kyungmin's user avatar
1 vote
1 answer
322 views

Is Metropolis-Hastings ever more efficient than rejection sampling in 2 dimensions?

I know that Metropolis-Hastings is an MCMC (Markov Chain Monte Carlo) method that is very useful in higher dimensions. The advantages it has over something like simple rejection sampling are that ...
Aditya S's user avatar
0 votes
1 answer
126 views

Computational aspect of the Metropolis-Hastings algorithm

One of the examples online is about how to write the Metropolis-Hastings algorithm from scratch. This tutorial uses a linear regression model as an example. Estimate three parameters: an intercept, a ...
Keith Lau's user avatar
  • 115
0 votes
0 answers
74 views

Metropolis Hastings Algorithms: How to measure the performance of algorithms? (Multidimensional)

I am working on a project and I am trying to measure the performance and compare two MCMC algorithms. The one is Random-Walk MH and the second one is PCN. I thought of maybe comparing the mean ...
makala's user avatar
  • 11
2 votes
2 answers
425 views

MCMC - How to derive the acceptance ratio from Markov chain detailed balance?

Please explain how the acceptance ratio: $$\rm A(x\to y) =\min\left(1,\frac{P(y)q(y\to x) }{P(x) q(x\to y) }\right)$$ is derived from the detailed balance: $$\rm \frac{A(x\to y) }{A(y\to x)}=\frac{P(y)...
mon's user avatar
  • 1,548
2 votes
0 answers
115 views

The guidelines for choosing different MCMC algorithms [closed]

MCMC has several types of algorithms: Metropolis-Hastings, Gibbs, Adaptive MH, Hamiltonian Monte Carlo. What are their respective pro/cons, and how to choose them in the Bayesian analysis?
user3269's user avatar
  • 5,222
3 votes
1 answer
500 views

Why do we sample from the uniform distribution in Metropolis-Hastings for acceptance?

For each iteration of the MH, sample $x'=q(x|x')$, then the acceptance probability is computed:$$A=\min(1,a)$$ where $$ \alpha=\frac{p(x')q(x|x')}{p(x)q(x'|x)} $$ Now, I've seen that the algorithm ...
wd violet's user avatar
  • 777
0 votes
1 answer
169 views

How can I find the acceptance probability for a joint Metropolis-Hastings proposal?

Suppose that I want to generate a proposal $(x^*,y^*,z^*)$ with the following: $$z^*\sim p(z|\alpha,\beta)$$ $$x^*\sim p(x|z^*,\boldsymbol{\gamma}_x)$$ $$y^*\sim p(y|z^*,\boldsymbol{\gamma}_y),$$ ...
Oski's user avatar
  • 5
2 votes
1 answer
380 views

How to understand the scaling in Metropolis Hastings MCMC

We know the Metropolis Hastings (MH) in MCMC: target distribution: $\pi(x)$ proposal distribution: $p(y|x)$ acceptance: $\alpha(x,y) = \min \Big(1, \dfrac{\pi(y)p(y|x)}{\pi(x)p(x|y)}\Big)$ Here are ...
user6703592's user avatar
  • 1,345
2 votes
0 answers
46 views

How to apply MCMC to bayes when likelihood is not easy to compute

Let $z$ be observations and $w$ be the parameter that we want to infer. Assuming that we know the prior $p(x)$, by using Bayes law, we have $p(x|z) = p(z|x)p(x)/p(z)$ where $Z$ is the marginal ...
orematasaburo's user avatar
4 votes
1 answer
93 views

Mean acceptance rate for Metropolis-Hastings algorithm

My question relates to the result stated on page 4 of: http://stat.columbia.edu/~gelman/research/published/baystat5.pdf which claims that the mean acceptance probability when performing the Metropolis-...
Matt Barrett's user avatar
1 vote
1 answer
2k views

Metropolis Hastings for Poisson Distribution

Studying Bayesian Inference and Markov chain Monte Carlo (MCMC) algorithms, I am facing a self study question on a MCMC approach to a Poisson distribution with parameter $\lambda$. Using R, my code is:...
Arduin's user avatar
  • 139
1 vote
0 answers
527 views

How can I tune my Random Walk Metropolis Hastings algorithm on the fly?

I just have a very general question. I've recently started to use Random Walk Metropolis-Hastings (RWMH) to sample from a distribution in order to calculate integrals. But I've noticed that the ...
AlphaBetaGamma96's user avatar
0 votes
1 answer
284 views

How do we define the kernel to calculate the acceptance ratio for Metropolis-Hastings Markov Chain Monte Carlo?

I am having a lot of difficulty understanding how to apply the algorithm to a real scenario. The part that confuses me is that we are looking for a target distribution (the real distribution of our ...
Radu's user avatar
  • 1
1 vote
0 answers
579 views

what is the optimal step size for metropolis-hastings algorithm to have independent state

In the PRML chapter 11, The Metropolis-Hasting algorithm, For a sampler with Gaussian distribution as proposal distribution. The original distribution is correlated multivariate Gaussian distribution, ...
Kelvin_neverKnow's user avatar
1 vote
0 answers
79 views

Metropolis-Hastings algorithm for a completely specified distribution

Consider a random variable $X\sim f(x)$, such that $$ f(x)=\frac{1}{c}\times K(x)\propto K(x), $$ where c: normalizing constant, K(x): the kernel of the distribution (ie the part which involves $x$). $...
Ludwig's user avatar
  • 63
1 vote
0 answers
406 views

Jacobian term for Metropolis Hastings algorithm?

Suppose an acceptance ratio of the MH sampler without parameter transformation is like this: \begin{gather} r=\frac{f( \theta ',\gamma '|y,m) q( \theta ',\gamma '|\theta ,\gamma )}{f( \theta ,\gamma |...
Keith Lau's user avatar
  • 115
1 vote
0 answers
136 views

Adaptive MCMC with low acceptance rate

I'm running an MCMC scheme defined in A tutorial on adaptive MCMC. Christophe Andrieu·Johannes Thoms. Stat Comput (2008) 18: 343–373DOI 10.1007/s11222-008-9110-y. They say the optimal scaling factor ...
user112495's user avatar
1 vote
2 answers
293 views

Simulated annealing acceptance probability puzzle

My understanding of simulated annealing (SA) is that at any iteration $t$, a new sample $Y_t$ is generated, which, if the objective function $E$ is improved, i.e., $E(Y_t)<E(X_{t-1})$, then $Y_t$ ...
user138668's user avatar
1 vote
1 answer
149 views

How do we need to define the acceptance ratio of the Metropolis-Hastings algorithm for a vanishing proposal density?

I've seen that different authors define the acceptance probability $\alpha$ of the Metropolis-Hastings algorithm with target distribution density $p$ and proposal kernel density $q$ differently; some ...
0xbadf00d's user avatar
  • 303
0 votes
1 answer
281 views

How do I account for numerical overflow with Adaptive MCMC?

EDIT: I tested Forgottenscience's solution below and it works; however, note that I found the working acceptance criterion to be that if $\log\alpha \geq \log u$, the point is accepted, where $u\sim\...
SandwichTheorem's user avatar
3 votes
1 answer
922 views

Do I need to evaluate acceptance rates in Metropolis within Gibbs algorithm?

Consider the Gibbs sampler Sample $\theta' \sim p(\theta|\tau, D)$ Sample $\tau' \sim p(\tau|\theta', D)$ where $\theta,\tau$ parameters of the data $D$. Now assume that we can only sample from $p(\...
tomka's user avatar
  • 6,624
1 vote
1 answer
103 views

Can a Metropolis-Hastings estimator converge if the proposal density vanishes whenver the target density vanishes?

I'm running the Metropolis-Hastings algorithm for a target and proposal density $p$ and $q$, respectively, for which it holds $$p(y)=0\Rightarrow q(x,y)=0\;\;\;\text{for all }y.\tag1$$ By definition ...
0xbadf00d's user avatar
  • 303
3 votes
1 answer
106 views

Erroneous expression for Metropolis-Hastings acceptance ratio in a paper

Let $(E,\mathcal E)$ be a measure space; $\rho:E\to[0,\infty)$ be $\mathcal E$-measurable, $p:E^2\to[0,\infty)$ be $\mathcal E^{\otimes2}$-measurable, $$r(x,y):=\left.\begin{cases}\displaystyle\frac{\...
0xbadf00d's user avatar
  • 303
3 votes
2 answers
577 views

Need to understand a statement for Random Walk Metropolis algorithm's proposal distribution?

I was told that the proposal distribution of Random Walk Metropolis needs to be symmetric. But today I was reading a book about Bayesian Analysis which contains the following statement: "The proposal ...
xabzakabecd's user avatar
  • 3,505
3 votes
1 answer
293 views

How to tune MCMC with unwieldy posterior [duplicate]

Let's say I have $n$ observations of a random variable, $X_1, \dotsm, X_n \sim \mathcal{N}(0, \sigma^2)$. I also assume $\sigma^2$ has a Gamma(1,1) prior distribution, $\pi(x) = \exp(-x)$. I'm now ...
CLL's user avatar
  • 229
1 vote
0 answers
221 views

convergence and efficiency of mcmc chains and estimation of covariance matrix

I am doing some bayesian analysis and exploring posterior distribution with mcmc method. I would like some clarification with estimating the covariance matrix. I have a model with 6 parameters. ...
simona's user avatar
  • 119
3 votes
1 answer
277 views

Expression for the mean acceptance rate of the Metropolis-Hastings algorithm

Let $(E,\mathcal E,\lambda)$ be a measure space $p:E\to[0,\infty)$ be $\mathcal E$-measurable with $$c:=\int p\:{\rm d}\lambda\in(0,\infty)$$ and $$\mu:=\underbrace{\frac1cp}_{=:\:\tilde p}\lambda$$ $...
0xbadf00d's user avatar
  • 303
1 vote
0 answers
513 views

MCMC Metropolis-Hastings sampler - Estimation of multiple parameters

First time that I ask a question on this platform! Here I go... I have a dataset with two random variables X1 and X2 and an output Y which comes from a discrete Weibull distribution. I've been trying ...
StevenVDL's user avatar
2 votes
0 answers
153 views

Why volume preservation is important for Metropolis update? [duplicate]

I think my question is naive but I would like to ask why why volume preservation is important for MCMC and specifically Metropolis update.I'm reading the following paper https://arxiv.org/pdf/1206....
G1I2A's user avatar
  • 85
6 votes
1 answer
167 views

How does the celebrated result about the diffusion limit of the Random Walk Metroplis-Hastings algorithm help us to find the optimal scaling

Let $d\in\mathbb N$ with $d>1$ $\ell>0$ $\sigma_d^2:=\frac{\ell^2}{d-1}$ $f\in C^2(\mathbb R)$ be positive with $$\int f(x)\:{\rm d}x=1$$ and $g:=\ln f$ $Q_d$ be a Markov kernel on $(\mathbb R^...
0xbadf00d's user avatar
  • 303
1 vote
1 answer
266 views

Proposal in MCMC lives in bigger space than parameter space. Which transformations should I choose?

I'm using a MCMC algorithm. The proposal is, due to lack of information on my part, a multivariate T-Student distribution, i.e. $\theta \sim \mathcal{MT}(\mu, \Sigma)$. However, some of the components ...
An old man in the sea.'s user avatar
4 votes
0 answers
195 views

Is the MC produced by HMC reversible?

I know that the deterministic dynamics in Hamiltonian Monte Carlo/Hybrid Monte Carlo are reversible and the numerical integrators one uses to approximate them are reversible too. But HMC consists of 2 ...
Lochend's user avatar
  • 51
1 vote
1 answer
147 views

2 versions of Metropolis-Hastings : are they equivalent?

I have seen 2 different versions of Metropolis algorithm. First one : Second one : I don't understand the differences between the 2 versions, especially in the second one where I have to use the ...
user avatar
2 votes
1 answer
3k views

Metropolis Hastings - Acceptance ratio, proposal and lkelihood

From a previous post : First to explain the MH algorithm, it's used to approximate numerically a target distribution, in this case $p(\theta|D)$. At each stage of the algorithm: A value ...
user avatar
5 votes
1 answer
2k views

How worried should I be about low acceptance rate in cold chain (parallel tempering MCMC sampler)

I have a very noisy/multimodal likelihood function for a 6-parameter model. The popular emcee sampler fails miserably (no matter how many chains I use and for how ...
Gabriel's user avatar
  • 4,352