I am working with a rather noisy and multi-modal likelihood. I've found that in order to obtain reasonable results from my Bayesian MCMC sampler (
emcee, an affine invariant MCMC ensemble sampler), I need to scale the log-likelihood "on the fly" to ensure the acceptance rate is within the 0.25-0.5 range.
The steps in pseudo-code are:
# Initial log-likelihood scale factor lkl_scale = 1. # Run the MCMC N times for i in 1..N # One sample from the MCMC sampler, passing the log-likelihood # and log-prior functions, and the log-likelihood scale factor sample = mcmc_sampler(log_lkl, log_prior, lkl_scale) # Obtain the acceptance rate for this step using some function accpt_rate = accpt_rate_func() if accpt_rate < 0.25 # If the acceptance rate is too low, reduce the scale factor lkl_scale = lkl_scale * 0.5 else if accpt_rate > 0.5 # If it is too large, increase it lkl_scale = lkl_scale * 2.
sample = mcmc_sampler(log_lkl, log_prior, lkl_scale) basically returns the sampled $\theta_i$ parameters and the associated
lkl_scale * log_lkl + log_prior values (for each chain) for this step.
lkl_scale factor scales the log-likelihood simply as
lkl_scale * log_lkl. This way if
lkl_scale is small, it will flatten the log-likelihood which results in larger acceptance rate values, and vice-versa. Notice I do not modify the priors ever.
I've been advised in a previous question that this approach is correct, and that it is similar to annealing or parallel tempering. But this method as far as I understand is based on "swapping between multiple Markov chains run in parallel at different temperatures to accelerate sampling" (Gupta et al. 2018) which I am not doing.
In another question it is stated that if you "flatten" the posterior, then you are effectively sampling from a different posterior and your samples need to be weighted to make sense. Since I am only modifying the log-likelihood, I'm not sure this applies to my case.
My question is then: is this approach statistically reasonable? If not, then: what if I were to scale my log-likelihood just once before launching the MCMC sampler (using a value that I know produces good results) Would it be reasonable then?