# Why does the redundant mean parameterization speed up Gibbs MCMC?

In Gelman & Hill (2007)'s book (Data Analysis Using Regression and Multilevel/Hierarchical Models), the authors claim that including redundant mean parameters can help speed up MCMC.

The given example is a non-nested model of "flight simulator" (Eq 13.9):

\begin{align} y_i &\sim N(\mu + \gamma_{j[i]} + \delta_{k[i]}, \sigma^2_y) \\ \gamma_j &\sim N(0, \sigma^2_\gamma) \\ \delta_k &\sim N(0, \sigma^2_\delta) \end{align}

They recommend a reparameterization, adding the mean parameters $\mu_\gamma$ and $\mu_\delta$ as follows:

\begin{align} \gamma_j \sim N(\mu_\gamma, \sigma^2_\gamma) \\ \delta_k \sim N(\mu_\delta, \sigma^2_\delta) \end{align}

The only justification offered is that (p. 420):

It is possible for the simulations to get stuck in a configuration where the entire vector $\gamma$ (or $\delta$) is far from zero (even though they are assigned a distribution with mean 0). Ultimately, the simulations will converge to the correct distribution, but we do not want to have to wait.

How do the redundant mean parameters help with this problem?

It seems to me that the non-nested model is slow mainly because of $\gamma$ and $\delta$ are negatively correlated. (Indeed, if one goes up, the other has to go down, given that their sum is "fixed" by the data). Do the redundant mean parameters help with reducing the correlation between $\gamma$ and $\delta$, or something else entirely?

• Are you looking for intuitive insight on this particular problem (e.g. whether it is the correlation $\gamma$ - $\delta$ or the correlations $\gamma$ - $\mu$ and $\delta$ - $\mu$), or are you looking for intuitive insight on the general problem (ie. the concept of hierarchical centering)? In the latter case, would you desire intuition that is close to a proof or intuition that is much more loose and shows more or less how it works? Commented Jan 23, 2018 at 13:01
• I'd like intuitive insight on the concept of hierarchical centering in general (since the particular case in the question is directly an application of hierarchical centering). The key point I want insight on is: why does hierarchical centering work if the variance at the group level is a considerable part of the total variance? The paper by Gelfand et al. proves this mathematically (i.e. derive the correlation and find its limiting behavior), but without any intuitive explanation. Commented Jan 23, 2018 at 22:23

The correlation to be avoided is the one between $\mu$ and the $\gamma_j$ and $\delta_k$.
By replacing $\gamma_j$ and $\delta_k$ in the computational model with alternative parameters that center around $\mu$ the correlation is reduced.