I'm seeking clarification regarding notation for Bayesian hierarchical models, specifically the mixed effects model. Consider the following hierarchical model for the outcome of unit $i \in N$ in group $j \in J$ from Gelman and Hill (p.265):

$$y_i \sim Normal(X_i\beta + \eta_{j[i]}, \sigma_y^2) \\ \eta_j \sim Normal(0, \sigma_\alpha^2)$$

  • $X_{(0)}$ is a constant.
  • $X_{(1)}$ is a unit/level-1 predictor.
  • $X_{(2)}$ is a group/level-2 predictor.

The authors call this the "regression with multiple error terms" notation but I find this confusing for several reasons.

  1. In this model isn't there only one error term ($\sigma_y^2$) that captures the total unexplained error in $y$? What then is the other error term in the name "multiple error terms?"

  2. If each group is supposed to have its own error, why is $\eta_j$ part of the model for the mean of $y_i$ rather than the error of $y_i$? I read this to imply a heteroskedastic model in which the error is decomposed into group-level terms and individual terms like $\sigma_y^2 = \sigma_\eta^2 + \sigma_i^2$.

  3. What, if anything, is multivariate here? Would it be more appropriate to write $\eta_j \sim MVN(0, \Sigma)$? If so, would $\Sigma$ have a single parameter $\sigma_\alpha^2$ on the diagonal and zeroes elsewhere or does each element of the diagonal vary (i.e. is its own parameter)?


1 Answer 1


Let's consider a simplification of your model with only a constant in the $X_i \beta$ part, i.e., $$\left\{ \begin{array}{l} y_{ji} \mid \eta_{j} \sim N(\eta_j, \sigma_y^2),\\ \eta_{j} \sim N(\beta, \sigma_\eta^2). \end{array} \right.$$ Note that I have explicitly conditioned on $\eta_j$ in the first line. That is, the first line represents the model at a given level $j$ $(j = 1, \ldots, n)$. The variance parameter $\sigma_y^2$ denotes the variance of the outcome $y_{ij}$ around the mean at the $j$-level denoted by $\beta + \eta_j$. The second line is the model for the means $\eta_1, \ldots, \eta_{n}$ that has a mean $\beta$ and variance $\sigma_{\eta}^2$.

From the above formulation, you can derive the marginal model for $y_j$, i.e., $$y_j \sim N(\beta, \Sigma),$$ where $\Sigma$ has a compound symmetry structure, with diagonal elements equal to $\sigma_{\eta}^2 + \sigma_y^2$, and off-diagonal elements equal to $\sigma_{\eta}^2$. This marginal model does not presume the existence of the latent variable $\eta_j$. That is, a hierarchical model implies a marginal model but not vice versa. For example, you could define a marginal model that does not correspond to a hierarchical one, or you could have that two different hierarchical models imply the same marginal model. You may have a look at Section 3.3 of my course notes for additional explanation.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.