I'm new to Markov Chain Monte Carlo (MCMC). Various sources suggest investigating the joint distributions of all pairs of esimated parameters. Refer to the figure below for an example. alpha and beta[4] are clearly more correlated than other pairs. Sources would suggest reparameterizing alpha and beta[4] in this case but I cannot find detailed explanation of the reasons why alpha and beta[4] being correlated could cause problems and what harms would alpha and beta[4] being correlated bring to the results. Could anyone provide an explanation or point me to related references?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ Give us some context, where and how you are using this and why. $\endgroup$– user2974951Commented Aug 30, 2022 at 6:19
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
0
Correlation between model parameters isn't a bad thing per se but this correlation often leads to poor mixing of MCMC chains. Highly-correlated surfaces in some dimensions are difficult to traverse due to the Metropolis-Hastings algorithm's non-zero acceptance of low probability areas. Leaving a highly correlated space of high probability can be difficult to return to unless proposal distributions are carefully constructed or you have an adaptive framework guiding the chains.
In the extreme, you can almost think of this as stepping off a plateau into a vast empty plain - you're going to spend a bit of time wandering around on the small hills of that plain before getting back onto the plateau unless you have a good map or are catching yourself from falling off the edge of the plateau in the first place.
An alternative to re-parameterization (although a little dimensional analysis is rarely a bad idea) is to use principal component analysis to generate your proposal distribution, as demonstrated in this paper. The idea is to have a good understanding of how your parameters correlate prior to running MCMC, and then use PCA to avoid the problem of correlation because principal components of your parameter sets aren't correlated.
