I'm using an adaptive MCMC (metropolis-hastings) scheme to infer some parameters. I've run 7 chains, each starting from a random point.
6 of the chains vaguely converge to the same area, but one of the chains converges to a different value, and with a much smaller variance.
The data I generated was synthetic, so I know that the other 6 chains were converging to the right area, and this other chain is converging to the wrong area. But what is this indiciative of? Or how can I determine what this is indicative of? Have I just got stuck in a local maxima somewhere? If so, how do I overcome this? Or is that the whole point of using multiple chains?
Another potential issue is that I'm currently using a diagonal covariance matrix. The parameters are biological, and so are probably correlated in some way? I'm aware that this is an issue I should probably correct, but could this specifically be causing this wrong convergence issue?