# How to reduce autocorrelation in MCMC

I'm using MCMC to simulation the distribution of some parameters in a Bayesian hierarchical model, which has the following form: $$\gamma_{ik} \sim Ber(\omega_{ik}).$$ Then I make a logit-transiformation, more specifically, $\theta_{ik}$ = $log(\frac{\omega_{ik}}{1-\omega_{ik}}).$

In order to incorporate covariates $x_{i}$, I use the following model, \begin{align} \left( \begin{array}{ccc} \theta_{i1} \\ \vdots \\ \theta_{iK} \end{array} \right)&=\mu + \beta x_{i} + \left( \begin{array}{c} \epsilon_{i1} \\ \vdots \\ \epsilon_{iK} \end{array} \right) \qquad i = 1,\ldots, I \\ \theta_{i} &\sim N(\mu + \beta x_{i},\Sigma) \\ \mu &\sim N(0,G) \\ \beta_{K \times p} &\sim MN(0,\Sigma,I) \\ \end{align} where $MN$ denotes the multivariate normal distribution.

I try to update these parameters separately, but it results in high autocorrelation. See the figure below,

Then I find out since $\mu$ and $\beta$ are not conditional independent given the other parameters, I should do block Gibbs sampling to avoid high autocorrelation between them. In order to do so, I need to calculate the covariance of $\beta$ and $\mu$ given the other parameters.

I tried to do this in the following way: convert $\beta$ into a vector vec($\beta$) with length $K \times p$, but I failed to calculate cov($\mu$ , $\beta$ $\mid \cdots$).

My question is

1: Is there any methods that I can use? 2: Is there any way to avoid high autocorrelation in this model? Any idea or suggestion would be appreciated.

• Are you asking about how to tune a specific MCMC algorithm? If so you should probably tell us which one you're trying. Also, what's your reasoning behind including the $\epsilon$s? – Taylor Aug 19 '18 at 19:36
• @Taylor I included $\epsilon$ as noise, which follows a normal distribution and this leads $\theta \sim normal$ . I'm going to update all the parameter except $\epsilon$ – Haotian Lin Aug 19 '18 at 20:36
• If you didn't have that noise, it would be a standard Bayesian logistic regression. Just some other thoughts off the top of my head: if you're using RWMH, try tuning your proposal distribution. Also, try independent MH. – Taylor Aug 21 '18 at 18:33