3
$\begingroup$

Say we are trying to estimate the posterior over a set of random variables using MCMC for a Bayesian model. We have prior knowledge about the variables and we can express this knowledge as a prior pdf.

Now, say this pdf admits (non-trivially) multiple parameterizations (i.e. different mathematical formulas that render the same exact pdf). Do MCMC solvers exploit in practice the parameterization chosen for the prior in any way? And would those differences respond to theoretical or technical reasons?

$\endgroup$
  • 2
    $\begingroup$ Certainly the way you parameterize your model (not just your prior!) can make a difference to the efficiency of some MCMC implementations $\endgroup$ – Glen_b Mar 3 '16 at 1:28
  • $\begingroup$ This seems to be a software-dependent question. One can certainly imagine two flavors of software, one which is somewhat naive and just runs the model-as-written and one which tries to sort out whether the efficiency might be improved by reconsidering the choice of parameterizations... Speaking about "MCMC solvers" in the abstract won't be answerable. $\endgroup$ – Sycorax Mar 3 '16 at 2:58
2
$\begingroup$

I am not aware of any MCMC method which would be parametrisation invariant (maybe someone more aware of the literature could validate this). In practice, the performance of for example Gibbs may be highly dependent from the working parametrisation : parametrisation involving highly a posteriori correlated (or dependent) variables will perform very badly while the one involving independent variables will give better results. For an illustration of why this is the case see e.g. http://wangxiaois.me/blog/how-to/2015/09/03/Gibbs_Sampling.html. I should even say that before implementing a model in Jags or anything else, it is a good practice to check for alternative parametrisation (at least, this is a thing I try to do and that sometimes save me hours of computing).

$\endgroup$
1
$\begingroup$

Yes, this can be the case. One classic case is over-parametrizing the prior on a hierarchical scale parameter.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.