Questions tagged [mcmc-acceptance-rate]
The acceptance rate (acceptance ratio, acceptance fraction) for a Markov Chain Monte Carlo sampler indicates the fraction of accepted over proposed moves.
41
questions
0
votes
0
answers
10
views
MCMCglmm package [closed]
the summary() for my ordinal model isn't returning all of my cutpoints. There are four levels in my response variable but I'm only getting two cut points. Does anyone know why?
- 1
0
votes
1
answer
30
views
Jacobian and proposal ratio of Birth/death step in RJMCMC of Gaussian mixture model
I am asking questions regarding RJMCMC several times in this site. Some of my questions are answered and some are unanswered. It didn't clarify all of my unclear points but I am glad that I have ...
- 45
1
vote
1
answer
47
views
Is Metropolis-Hastings ever more efficient than rejection sampling in 2 dimensions?
I know that Metropolis-Hastings is an MCMC (Markov Chain Monte Carlo) method that is very useful in higher dimensions. The advantages it has over something like simple rejection sampling are that ...
- 11
0
votes
0
answers
44
views
Random scan Gibbs sampling as special case of Metropolis-Hastings
I am reading Blitzstein's Introduction to Probability and come across with the following proof that I don't really understand:
Theorem: The random scan Gibbs sampler is a special case of the ...
0
votes
0
answers
12
views
Is it possible to increase the Hastings ratio by combining and mixing elementary kernels?
Let's say I am working with a state $X$ split into three parts $U$, $V$, and $W$.
I can efficiently sample from $W|U,V$, $U|V$, and $V|U$. My initial intuition was to do a variable-at-a-time ...
- 53
0
votes
1
answer
84
views
Computational aspect of the Metropolis-Hastings algorithm
One of the examples online is about how to write the Metropolis-Hastings algorithm from scratch. This tutorial uses a linear regression model as an example. Estimate three parameters: an intercept, a ...
- 105
0
votes
0
answers
42
views
Metropolis Hastings Algorithms: How to measure the performance of algorithms? (Multidimensional)
I am working on a project and I am trying to measure the performance and compare two MCMC algorithms. The one is Random-Walk MH and the second one is PCN.
I thought of maybe comparing the mean ...
- 11
2
votes
2
answers
163
views
MCMC - How to derive the acceptance ratio from Markov chain detailed balance?
Please explain how the acceptance ratio:
$$\rm A(x\to y) =\min\left(1,\frac{P(y)q(y\to x) }{P(x) q(x\to y) }\right)$$
is derived from the detailed balance:
$$\rm \frac{A(x\to y) }{A(y\to x)}=\frac{P(y)...
- 1,344
0
votes
0
answers
18
views
Metropolis acceptance ratio when working with logs [duplicate]
We wish to sample from a posterior distribution, $\pi(\theta | y)$ using a metropolis hastings MCMC algorithm. The analytical form of this is distribution is proportional to the likelihood function $\...
- 1
2
votes
0
answers
70
views
The guidelines for choosing different MCMC algorithms [closed]
MCMC has several types of algorithms: Metropolis-Hastings, Gibbs, Adaptive MH, Hamiltonian Monte Carlo. What are their respective pro/cons, and how to choose them in the Bayesian analysis?
- 5,084
0
votes
0
answers
31
views
Why Reversible jump mcmc has only one step increase/ decrease?
I was applying reversible jump MCMC for joint estimation of model order and parameter estimation. I've a conceptual question in my mind. First of all, the algorithm has 3 steps, namely the birth, ...
- 113
3
votes
1
answer
305
views
Why do we sample from the uniform distribution in Metropolis-Hastings for acceptance?
For each iteration of the MH, sample $x'=q(x|x')$, then the acceptance probability is computed:$$A=\min(1,a)$$
where
$$
\alpha=\frac{p(x')q(x|x')}{p(x)q(x'|x)}
$$
Now, I've seen that the algorithm ...
- 601
0
votes
1
answer
115
views
How can I find the acceptance probability for a joint Metropolis-Hastings proposal?
Suppose that I want to generate a proposal $(x^*,y^*,z^*)$ with the following:
$$z^*\sim p(z|\alpha,\beta)$$
$$x^*\sim p(x|z^*,\boldsymbol{\gamma}_x)$$
$$y^*\sim p(y|z^*,\boldsymbol{\gamma}_y),$$
...
- 5
2
votes
1
answer
305
views
How to understand the scaling in Metropolis Hastings MCMC
We know the Metropolis Hastings (MH) in MCMC:
target distribution: $\pi(x)$
proposal distribution: $p(y|x)$
acceptance: $\alpha(x,y) = \min \Big(1, \dfrac{\pi(y)p(y|x)}{\pi(x)p(x|y)}\Big)$
Here are ...
- 1,245
2
votes
0
answers
44
views
How to apply MCMC to bayes when likelihood is not easy to compute
Let $z$ be observations and $w$ be the parameter that we want to infer. Assuming that we know the prior $p(x)$, by using Bayes law, we have $p(x|z) = p(z|x)p(x)/p(z)$ where $Z$ is the marginal ...
- 211
4
votes
1
answer
91
views
Mean acceptance rate for Metropolis-Hastings algorithm
My question relates to the result stated on page 4 of: http://stat.columbia.edu/~gelman/research/published/baystat5.pdf
which claims that the mean acceptance probability when performing the Metropolis-...
- 41
1
vote
1
answer
1k
views
Metropolis Hastings for Poisson Distribution
Studying Bayesian Inference and Markov chain Monte Carlo (MCMC) algorithms, I am facing a self study question on a MCMC approach to a Poisson distribution with parameter $\lambda$.
Using R, my code is:...
- 139
1
vote
0
answers
367
views
How can I tune my Random Walk Metropolis Hastings algorithm on the fly?
I just have a very general question. I've recently started to use Random Walk Metropolis-Hastings (RWMH) to sample from a distribution in order to calculate integrals. But I've noticed that the ...
- 248
0
votes
1
answer
222
views
How do we define the kernel to calculate the acceptance ratio for Metropolis-Hastings Markov Chain Monte Carlo?
I am having a lot of difficulty understanding how to apply the algorithm to a real scenario.
The part that confuses me is that we are looking for a target distribution (the real distribution of our ...
- 1
1
vote
0
answers
455
views
what is the optimal step size for metropolis-hastings algorithm to have independent state
In the PRML chapter 11, The Metropolis-Hasting algorithm,
For a sampler with Gaussian distribution as proposal distribution.
The original distribution is correlated multivariate Gaussian distribution, ...
1
vote
0
answers
69
views
Metropolis-Hastings algorithm for a completely specified distribution
Consider a random variable $X\sim f(x)$, such that
$$
f(x)=\frac{1}{c}\times K(x)\propto K(x),
$$
where c: normalizing constant, K(x): the kernel of the distribution (ie the part which involves $x$). $...
- 63
1
vote
0
answers
341
views
Jacobian term for Metropolis Hastings algorithm?
Suppose an acceptance ratio of the MH sampler without parameter transformation is like this:
\begin{gather}
r=\frac{f( \theta ',\gamma '|y,m) q( \theta ',\gamma '|\theta ,\gamma )}{f( \theta ,\gamma |...
- 105
1
vote
0
answers
114
views
Adaptive MCMC with low acceptance rate
I'm running an MCMC scheme defined in A tutorial on adaptive MCMC. Christophe Andrieu·Johannes Thoms. Stat Comput (2008) 18: 343–373DOI 10.1007/s11222-008-9110-y.
They say the optimal scaling factor ...
- 317
1
vote
2
answers
242
views
Simulated annealing acceptance probability puzzle
My understanding of simulated annealing (SA) is that at any iteration $t$, a new sample $Y_t$ is generated, which, if the objective function $E$ is improved, i.e., $E(Y_t)<E(X_{t-1})$, then $Y_t$ ...
- 121
1
vote
1
answer
147
views
How do we need to define the acceptance ratio of the Metropolis-Hastings algorithm for a vanishing proposal density?
I've seen that different authors define the acceptance probability $\alpha$ of the Metropolis-Hastings algorithm with target distribution density $p$ and proposal kernel density $q$ differently; some ...
- 71
0
votes
1
answer
252
views
How do I account for numerical overflow with Adaptive MCMC?
EDIT: I tested Forgottenscience's solution below and it works; however, note that I found the working acceptance criterion to be that if $\log\alpha \geq \log u$, the point is accepted, where $u\sim\...
3
votes
1
answer
794
views
Do I need to evaluate acceptance rates in Metropolis within Gibbs algorithm?
Consider the Gibbs sampler
Sample $\theta' \sim p(\theta|\tau, D)$
Sample $\tau' \sim p(\tau|\theta', D)$
where $\theta,\tau$ parameters of the data $D$. Now assume that we can only sample from $p(\...
- 6,382
1
vote
1
answer
97
views
Can a Metropolis-Hastings estimator converge if the proposal density vanishes whenver the target density vanishes?
I'm running the Metropolis-Hastings algorithm for a target and proposal density $p$ and $q$, respectively, for which it holds $$p(y)=0\Rightarrow q(x,y)=0\;\;\;\text{for all }y.\tag1$$
By definition ...
- 71
3
votes
1
answer
105
views
Erroneous expression for Metropolis-Hastings acceptance ratio in a paper
Let
$(E,\mathcal E)$ be a measure space;
$\rho:E\to[0,\infty)$ be $\mathcal E$-measurable, $p:E^2\to[0,\infty)$ be $\mathcal E^{\otimes2}$-measurable, $$r(x,y):=\left.\begin{cases}\displaystyle\frac{\...
- 71
3
votes
2
answers
446
views
Need to understand a statement for Random Walk Metropolis algorithm's proposal distribution?
I was told that the proposal distribution of Random Walk Metropolis needs to be symmetric. But today I was reading a book about Bayesian Analysis which contains the following statement:
"The proposal ...
- 3,145
3
votes
1
answer
278
views
How to tune MCMC with unwieldy posterior [duplicate]
Let's say I have $n$ observations of a random variable, $X_1, \dotsm, X_n \sim \mathcal{N}(0, \sigma^2)$. I also assume $\sigma^2$ has a Gamma(1,1) prior distribution, $\pi(x) = \exp(-x)$.
I'm now ...
- 229
1
vote
0
answers
203
views
convergence and efficiency of mcmc chains and estimation of covariance matrix
I am doing some bayesian analysis and exploring posterior distribution with mcmc method. I would like some clarification with estimating the covariance matrix. I have a model with 6 parameters. ...
- 119
3
votes
1
answer
265
views
Expression for the mean acceptance rate of the Metropolis-Hastings algorithm
Let
$(E,\mathcal E,\lambda)$ be a measure space
$p:E\to[0,\infty)$ be $\mathcal E$-measurable with $$c:=\int p\:{\rm d}\lambda\in(0,\infty)$$ and $$\mu:=\underbrace{\frac1cp}_{=:\:\tilde p}\lambda$$
$...
- 71
1
vote
0
answers
473
views
MCMC Metropolis-Hastings sampler - Estimation of multiple parameters
First time that I ask a question on this platform! Here I go...
I have a dataset with two random variables X1 and X2 and an output Y which comes from a discrete Weibull distribution.
I've been trying ...
- 11
2
votes
0
answers
152
views
Why volume preservation is important for Metropolis update? [duplicate]
I think my question is naive but I would like to ask why why volume preservation is important for MCMC and specifically Metropolis update.I'm reading the following paper https://arxiv.org/pdf/1206....
- 85
6
votes
1
answer
165
views
How does the celebrated result about the diffusion limit of the Random Walk Metroplis-Hastings algorithm help us to find the optimal scaling
Let
$d\in\mathbb N$ with $d>1$
$\ell>0$
$\sigma_d^2:=\frac{\ell^2}{d-1}$
$f\in C^2(\mathbb R)$ be positive with $$\int f(x)\:{\rm d}x=1$$ and $g:=\ln f$
$Q_d$ be a Markov kernel on $(\mathbb R^...
- 71
1
vote
1
answer
238
views
Proposal in MCMC lives in bigger space than parameter space. Which transformations should I choose?
I'm using a MCMC algorithm. The proposal is, due to lack of information on my part, a multivariate T-Student distribution, i.e. $\theta \sim \mathcal{MT}(\mu, \Sigma)$. However, some of the components ...
- 5,400
4
votes
0
answers
180
views
Is the MC produced by HMC reversible?
I know that the deterministic dynamics in Hamiltonian Monte Carlo/Hybrid Monte Carlo are reversible and the numerical integrators one uses to approximate them are reversible too. But HMC consists of 2 ...
- 51
1
vote
1
answer
132
views
2 versions of Metropolis-Hastings : are they equivalent?
I have seen 2 different versions of Metropolis algorithm.
First one :
Second one :
I don't understand the differences between the 2 versions, especially in the second one where I have to use the ...
user226073
2
votes
1
answer
2k
views
Metropolis Hastings - Acceptance ratio, proposal and lkelihood
From a previous post :
First to explain the MH algorithm, it's used to approximate
numerically a target distribution, in this case $p(\theta|D)$. At
each stage of the algorithm:
A value ...
user226073
3
votes
1
answer
1k
views
How worried should I be about low acceptance rate in cold chain (parallel tempering MCMC sampler)
I have a very noisy/multimodal likelihood function for a 6-parameter model. The popular emcee sampler fails miserably (no matter how many chains I use and for how ...
- 3,552