I am working on Parameter estimation. I've been told to use MCMC. I've chosen the Metropolis-Hasting algorithm. I think I understand the algorithm.

I have around 8 input parameters to estimate. How do I go about choosing a multivariate proposal distribution and acceptance ratio? what factors should be considered? Also, the values for parameters need to be varied only by +\- 50% of their default values, How can I specify this?

I have seen the code examples of MCMC in Python and R, I don't understand them that well, most of them are for 2-3 variables.

Any help is appreciated. It would also be sufficient if you could just explain me the intuition behind choosing the proposal distribution and the acceptance ratio function.

  • $\begingroup$ you don't choose the acceptance ratio. You calculate it at every iteration using your choice of proposal and model. $\endgroup$ – Taylor Jul 6 '17 at 19:27
  • $\begingroup$ @Taylor: the average acceptance ratio is governed by the proposal. Proposal that usually depends on parameters like scale. Hence by choosing the scale you can modify the average acceptance ratio. This is the essence of adaptive MCMC algorithms. $\endgroup$ – Xi'an Jul 12 '17 at 20:07
  • $\begingroup$ @Xi'an, I assumed OP was under the impression that both the proposal distribution and the acceptance ratio determined by it (and the model) were left up to choice. I was just trying to be helpful by reducing the number of things that have to be selected. Regarding adaptive MCMC: it's on my reading list, and I am not too familiar with it at the moment. $\endgroup$ – Taylor Jul 13 '17 at 2:40

Here's a quote from page 306 of the second edition of Bayesian Data Analysis that has helped me personally with the same question:

It is hard to give general advice on efficient jumping rules, but some results have been obtained for normal random walk jumping distribtutions that seem to be useful in many problems. Suppose there are $d$ parameters, and the posterior distribution $\theta = (\theta_1, \ldots, \theta_d)$, after appropriate transformation, is multivariate normal with known variance matrix $\Sigma$. Further suppose that we will take draws using the Metropolis algorithm with a normal jumping kernel centered on the current point and with the same shape as the target distribution: that is $J(\theta^*|\theta^{t-1}) = N(\theta^*|\theta^{t-1},c^2\Sigma)$. Among this class of jumping rules, the most efficient has scale $c \approx 2.4/\sqrt{d}$, where efficiency is defined relative to independent sampling from the posterior distributuon.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.