Questions tagged [mcmc-acceptance-rate]

The acceptance rate (acceptance ratio, acceptance fraction) for a Markov Chain Monte Carlo sampler indicates the fraction of accepted over proposed moves.

Filter by
Sorted by
Tagged with
0 votes
1 answer
31 views

How can I find the acceptance probability for a joint Metropolis-Hastings proposal?

Suppose that I want to generate a proposal $(x^*,y^*,z^*)$ with the following: $$z^*\sim p(z|\alpha,\beta)$$ $$x^*\sim p(x|z^*,\boldsymbol{\gamma}_x)$$ $$y^*\sim p(y|z^*,\boldsymbol{\gamma}_y),$$ ...
user avatar
  • 5
0 votes
0 answers
38 views

Help to Proof of Gibbs sampler acceptance rate

i was wondering if someone could explain how the acceptance rate in the Gibbs Sampling works. In the literature it says that in Gibbs case, the acceptance is always 1 because part of the "...
user avatar
2 votes
1 answer
83 views

How to understand the scaling in Metropolis Hastings MCMC

We know the Metropolis Hastings (MH) in MCMC: target distribution: $\pi(x)$ proposal distribution: $p(y|x)$ acceptance: $\alpha(x,y) = \min \Big(1, \dfrac{\pi(y)p(y|x)}{\pi(x)p(x|y)}\Big)$ Here are ...
user avatar
2 votes
0 answers
29 views

How to apply MCMC to bayes when likelihood is not easy to compute

Let $z$ be observations and $w$ be the parameter that we want to infer. Assuming that we know the prior $p(x)$, by using Bayes law, we have $p(x|z) = p(z|x)p(x)/p(z)$ where $Z$ is the marginal ...
user avatar
0 votes
0 answers
38 views

Estimates from MCMCglmm multivariate regression

I'm running a multivariate regression on two response variables (X and Y) using MCMCglmm in R. X is a continuous variable (family = Guassian) while Y is a binary response variable (family = threshold)....
user avatar
  • 21
4 votes
1 answer
66 views

Mean acceptance rate for Metropolis-Hastings algorithm

My question relates to the result stated on page 4 of: http://stat.columbia.edu/~gelman/research/published/baystat5.pdf which claims that the mean acceptance probability when performing the Metropolis-...
user avatar
1 vote
1 answer
644 views

Metropolis Hastings for Poisson Distribution

Studying Bayesian Inference and Markov chain Monte Carlo (MCMC) algorithms, I am facing a self study question on a MCMC approach to a Poisson distribution with parameter $\lambda$. Using R, my code is:...
user avatar
  • 139
1 vote
0 answers
45 views

How can I tune my Random Walk Metropolis Hastings algorithm on the fly?

I just have a very general question. I've recently started to use Random Walk Metropolis-Hastings (RWMH) to sample from a distribution in order to calculate integrals. But I've noticed that the ...
user avatar
0 votes
1 answer
93 views

How do we define the kernel to calculate the acceptance ratio for Metropolis-Hastings Markov Chain Monte Carlo?

I am having a lot of difficulty understanding how to apply the algorithm to a real scenario. The part that confuses me is that we are looking for a target distribution (the real distribution of our ...
user avatar
  • 1
1 vote
0 answers
248 views

what is the optimal step size for metropolis-hastings algorithm to have independent state

In the PRML chapter 11, The Metropolis-Hasting algorithm, For a sampler with Gaussian distribution as proposal distribution. The original distribution is correlated multivariate Gaussian distribution, ...
user avatar
1 vote
0 answers
39 views

Metropolis-Hastings algorithm for a completely specified distribution

Consider a random variable $X\sim f(x)$, such that $$ f(x)=\frac{1}{c}\times K(x)\propto K(x), $$ where c: normalizing constant, K(x): the kernel of the distribution (ie the part which involves $x$). $...
user avatar
  • 53
0 votes
0 answers
157 views

Jacobian term for Metropolis Hastings algorithm?

Suppose an acceptance ratio of the MH sampler without parameter transformation is like this: \begin{gather} r=\frac{f( \theta ',\gamma '|y,m) q( \theta ',\gamma '|\theta ,\gamma )}{f( \theta ,\gamma |...
user avatar
1 vote
0 answers
56 views

Adaptive MCMC with low acceptance rate

I'm running an MCMC scheme defined in A tutorial on adaptive MCMC. Christophe Andrieu·Johannes Thoms. Stat Comput (2008) 18: 343–373DOI 10.1007/s11222-008-9110-y. They say the optimal scaling factor ...
user avatar
1 vote
2 answers
151 views

Simulated annealing acceptance probability puzzle

My understanding of simulated annealing (SA) is that at any iteration $t$, a new sample $Y_t$ is generated, which, if the objective function $E$ is improved, i.e., $E(Y_t)<E(X_{t-1})$, then $Y_t$ ...
user avatar
1 vote
1 answer
104 views

How do we need to define the acceptance ratio of the Metropolis-Hastings algorithm for a vanishing proposal density?

I've seen that different authors define the acceptance probability $\alpha$ of the Metropolis-Hastings algorithm with target distribution density $p$ and proposal kernel density $q$ differently; some ...
user avatar
  • 539
0 votes
1 answer
170 views

How do I account for numerical overflow with Adaptive MCMC?

EDIT: I tested Forgottenscience's solution below and it works; however, note that I found the working acceptance criterion to be that if $\log\alpha \geq \log u$, the point is accepted, where $u\sim\...
user avatar
3 votes
1 answer
494 views

Do I need to evaluate acceptance rates in Metropolis within Gibbs algorithm?

Consider the Gibbs sampler Sample $\theta' \sim p(\theta|\tau, D)$ Sample $\tau' \sim p(\tau|\theta', D)$ where $\theta,\tau$ parameters of the data $D$. Now assume that we can only sample from $p(\...
user avatar
  • 5,934
1 vote
1 answer
87 views

Can a Metropolis-Hastings estimator converge if the proposal density vanishes whenver the target density vanishes?

I'm running the Metropolis-Hastings algorithm for a target and proposal density $p$ and $q$, respectively, for which it holds $$p(y)=0\Rightarrow q(x,y)=0\;\;\;\text{for all }y.\tag1$$ By definition ...
user avatar
  • 539
3 votes
1 answer
102 views

Erroneous expression for Metropolis-Hastings acceptance ratio in a paper

Let $(E,\mathcal E)$ be a measure space; $\rho:E\to[0,\infty)$ be $\mathcal E$-measurable, $p:E^2\to[0,\infty)$ be $\mathcal E^{\otimes2}$-measurable, $$r(x,y):=\left.\begin{cases}\displaystyle\frac{\...
user avatar
  • 539
2 votes
2 answers
224 views

Need to understand a statement for Random Walk Metropolis algorithm's proposal distribution?

I was told that the proposal distribution of Random Walk Metropolis needs to be symmetric. But today I was reading a book about Bayesian Analysis which contains the following statement: "The proposal ...
user avatar
  • 2,585
3 votes
1 answer
217 views

How to tune MCMC with unwieldy posterior [duplicate]

Let's say I have $n$ observations of a random variable, $X_1, \dotsm, X_n \sim \mathcal{N}(0, \sigma^2)$. I also assume $\sigma^2$ has a Gamma(1,1) prior distribution, $\pi(x) = \exp(-x)$. I'm now ...
user avatar
  • 229
1 vote
0 answers
130 views

convergence and efficiency of mcmc chains and estimation of covariance matrix

I am doing some bayesian analysis and exploring posterior distribution with mcmc method. I would like some clarification with estimating the covariance matrix. I have a model with 6 parameters. ...
user avatar
  • 119
3 votes
1 answer
241 views

Expression for the mean acceptance rate of the Metropolis-Hastings algorithm

Let $(E,\mathcal E,\lambda)$ be a measure space $p:E\to[0,\infty)$ be $\mathcal E$-measurable with $$c:=\int p\:{\rm d}\lambda\in(0,\infty)$$ and $$\mu:=\underbrace{\frac1cp}_{=:\:\tilde p}\lambda$$ $...
user avatar
  • 539
1 vote
0 answers
353 views

MCMC Metropolis-Hastings sampler - Estimation of multiple parameters

First time that I ask a question on this platform! Here I go... I have a dataset with two random variables X1 and X2 and an output Y which comes from a discrete Weibull distribution. I've been trying ...
user avatar
0 votes
0 answers
17 views

How to evaluate the draw from the proposal of a Metropolis-Hastings?

In the Metropolis-Hastings step of a MCMC, given a $\theta_n$, I'm drawing $\theta_{n+1} \sim F(\mu(\theta_n), \Sigma)$ where the $\mu $ is a location vector and $\Sigma$ is a scale matrix. When ...
user avatar
1 vote
0 answers
146 views

Why volume preservation is important for Metropolis update? [duplicate]

I think my question is naive but I would like to ask why why volume preservation is important for MCMC and specifically Metropolis update.I'm reading the following paper https://arxiv.org/pdf/1206....
user avatar
  • 75
6 votes
1 answer
148 views

How does the celebrated result about the diffusion limit of the Random Walk Metroplis-Hastings algorithm help us to find the optimal scaling

Let $d\in\mathbb N$ with $d>1$ $\ell>0$ $\sigma_d^2:=\frac{\ell^2}{d-1}$ $f\in C^2(\mathbb R)$ be positive with $$\int f(x)\:{\rm d}x=1$$ and $g:=\ln f$ $Q_d$ be a Markov kernel on $(\mathbb R^...
user avatar
  • 539
1 vote
1 answer
174 views

Proposal in MCMC lives in bigger space than parameter space. Which transformations should I choose?

I'm using a MCMC algorithm. The proposal is, due to lack of information on my part, a multivariate T-Student distribution, i.e. $\theta \sim \mathcal{MT}(\mu, \Sigma)$. However, some of the components ...
user avatar
3 votes
0 answers
138 views

Is the MC produced by HMC reversible?

I know that the deterministic dynamics in Hamiltonian Monte Carlo/Hybrid Monte Carlo are reversible and the numerical integrators one uses to approximate them are reversible too. But HMC consists of 2 ...
user avatar
  • 41
1 vote
1 answer
99 views

2 versions of Metropolis-Hastings : are they equivalent?

I have seen 2 different versions of Metropolis algorithm. First one : Second one : I don't understand the differences between the 2 versions, especially in the second one where I have to use the ...
user avatar
2 votes
1 answer
2k views

Metropolis Hastings - Acceptance ratio, proposal and lkelihood

From a previous post : First to explain the MH algorithm, it's used to approximate numerically a target distribution, in this case $p(\theta|D)$. At each stage of the algorithm: A value ...
user avatar
2 votes
1 answer
882 views

How worried should I be about low acceptance rate in cold chain (parallel tempering MCMC sampler)

I have a very noisy/multimodal likelihood function for a 6-parameter model. The popular emcee sampler fails miserably (no matter how many chains I use and for how ...
user avatar
  • 3,124