Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I used Metropolis-Hastings algorithm to run a MCMC to some failure data in Bayesian analysis and the posterior samples of the parameters were obtained. There's a strong correlation between the posterior samples. Does the correlation affect the point estimates from the posterior values? Can anyone please help me with the interpretation of the correlation between the posterior parameter values?
Thank you
You clarified that you're talking about a correlation between concurrent samples of different parameters. This is different from the serial correlation that the other answer goes into (which is why I asked). Serial correlation between samples of the same parameter arises in many MCMC algorithms, because of the "random walk" behavior of the algorithm, which means that each new sample is a finite step away from the current sample. But this random walk behavior does not cause the kind of correlation you're seeing, between concurrent samples of different parameters.
A correlation between samples of different parameters normally just means that the posterior distributions of those parameters are in fact correlated. E.g. say you have some data $y$ that is bivariate Normally distributed: $$ y\sim N\left(\begin{bmatrix} \mu_1\\ \mu_2 \end{bmatrix},\begin{bmatrix} \sigma_1^2 & \rho\sigma_1\sigma_2\\ \rho\sigma_1\sigma_2 & \sigma_2^2 \end{bmatrix} \right) $$
Then the posterior $p\left(\begin{bmatrix} \mu_1\\ \mu_2 \end{bmatrix}\mid y\right)$ will be correlated (between $\mu_1$ and $\mu_2$) in proportion to $\rho$, and therefore samples of $\boldsymbol{\mu}$ under this posterior will also be correlated with each other.
So unless you have reason to believe that the posteriors shouldn't be correlated, there is nothing to worry about (and to answer your question: your interpretation of this correlation should be that these posteriors are not indepedent). The point-estimates of your parameters won't be biased, at least in the first order (i.e. their values won't tend to be systematically over- or underestimated).
The serial correlation does not affect the mean values of the parameters, but the uncertainty on the mean value. You have to estimate the statistical efficiency (related to the serial correlation length) and use it to correct the number of samples. For a parameter $p$, the uncertainty on the mean becomes $\sigma/\sqrt{\rho*N}$ where $\sigma$ is the standard deviation of $p$, $0\le\rho\le1$ is the statistical efficiency and $N$ the sample size.
For instance, the function coda::effectiveSize() in R gives you directly the product $\rho*N$.
