I am training a complex Bayesian model using Gibbs sampling and Metropolis-Hasting algorithm. Most of the parameters are directly sampled by using conjugate priors except for 3 params which are sampled by M-H. In tuning the "step size" of random walk M-H, I found making the acceptance rate to be around 0.234~0.44 will only give 1%~5% effective samples (strong autocorrelations). May I ask is the "effective size" more "important" than the acceptance rate as a criterion? And which criterion should I "ultimately" care here?
$\begingroup$
$\endgroup$
4
-
$\begingroup$ What are the effective samples you get if acceptance rate it increased? $\endgroup$– GreenparkerSep 28 '16 at 12:53
-
1$\begingroup$ The effective samples has a lot more to do with the posterior correlation amongst your parameters than the acceptance rate of your MH steps, e.g. using a Gibbs sampler on a bivariate normal with high correlation will result in a Markov chain that has high correlation in your samples even though the acceptance rate is 1. $\endgroup$– jaradniemiSep 28 '16 at 14:41
-
1$\begingroup$ Aiming at this "optimal" acceptance rate in a regular random walk Metropolis-Hastings algorithm is equivalent to minimising the asymptotic variance of the resulting estimator, hence at reducing the autocorrelation in the chain. The effective sample size is a poor proxy with well-known shortcomings and is too easily adopted due to its simplicity. $\endgroup$– Xi'anSep 29 '16 at 20:02
-
$\begingroup$ I am quite new to this. Which paper/book are the best in discussing this topics? I am currently reading the source code of jags. It seems it just use acceptance rate as the criterion. And the adaptive alg is "noisy gradient" --- what is that about? I cannot find any references on this. $\endgroup$– user112758Sep 29 '16 at 20:29
Add a comment
|