I have an implementation of an MCMC algorithm (Metropolis-Hastings and Adaptive Metropolis-Hastings) that I want to modify to suit my needs (it's pyMC, if anyone is interested on the details).
My problem is that the parameter space for my model is very high (10-20 parameters), so the burn-in period is crucial for each MCMC chain to converge. I've read that for a multivariate gaussian posterior an acceptance rate of 0.234 is ok (Gelman et. al, 2004), but in order to get those acceptance rates I need to stabilize the variances of the gaussian jumps between proposal values of each of my parameters in this burn-in period: can someone shine some light on rules (and proofs!) on how to stabilize them?
So far I've only seen very few rules without proof. The standard rule for the adaptation of the variances in pyMC can be seen here, from lines 562 to 581, which is rather arbitrary. On the other hand, Ford (2006) on section 3.2 proposes to adapt the variances by some sort of rule-of-thumb which I never heard of before. I've also been thinking that by monitoring how the variance affects the acceptance rates one could detect the optimal variances for the jumps...