For the mixed effects (random slope) binary logistic regression model the logit can be expressed as (Applied Logistic Regression (2010) - David W,HOSMER, JR.):

$$g(\boldsymbol{x}_{i,j},\alpha_i,\boldsymbol{\beta}_s) = \beta_0 + \alpha_i + \boldsymbol{x}_{i,j} \boldsymbol{\beta}_s $$

Where $\alpha_i \sim N(0,\sigma^2_\alpha)$. and (i,j) refers to the jth observation in the i:th cluster. the subscript s in $\beta_s$ refers to the fact that coefficients refer to a model that is specific to subjects with random effect $\alpha_i$

This model is said to account for correlated observations by adding a random effect term ,specific to the cluster, to the logit.

I dont understand what is the effect of adding this "random term" to the model? Why is this model formultion better when having correlated observations?

'The hetrogenity in the $\alpha_i$'s is controlled by the varaince $\sigma^2_\alpha$.Thus, as $\sigma^2_\alpha$ increases the within cluster correlation increases'

I dont see the relation with the variance $\alpha_i$ and the correlation within each cluster. Why does greater variance imply greater within cluster correlation?


1 Answer 1


$\newcommand{\e}{\varepsilon}$$\newcommand{\one}{\mathbf 1}$$\newcommand{\Var}{\operatorname{Var}}$$\newcommand{\Cov}{\operatorname{Cov}}$It seems a lot of your question can be answered with just a random intercepts model so I'll start there. It also doesn't seem like this being a logistic regression particularly matters so I'll begin by considering the simple random intercepts model $$ y_{ij} = \mu + \alpha_i + \e_{ij} $$ with $\alpha\sim \mathcal N(\mathbf 0, \sigma^2_\alpha I) \perp \e \sim \mathcal N(\mathbf 0, \sigma^2 I)$. $i=1,\dots,m$ will index groups and $j=1,\dots,n_i$ will index observations within a group. There are $n = \sum_{i=1}^m n_i$ observations in all.

Note that $$ \Cov(y_{ij}, y_{kl}) = \begin{cases} \sigma^2 + \sigma^2_\alpha & i=j, k=l\\ \sigma^2_\alpha & i=j, k\neq l \\ 0 & i\neq j \end{cases} $$ so this reveals the effect of including $\alpha_i$, namely that it adds to the within-cluster covariance. As $\sigma^2_\alpha$ increases, especially with $\sigma^2$ fixed, the $y_{ij}$ are dominated by the $\alpha_i$ and this leads to the values varying relatively little within a cluster.

For a GLM, this'll still have the effect of making within-cluster predictions more similar than between-cluster, and the larger $\sigma^2_\alpha$ is, the more similar the predictions for units in the same cluster will be.

Now adding in a random slopes term, we'll have $$ y_{ij} = \mu + \alpha_i + (\beta + \gamma_i) x_{ij} + \e_{ij} $$ where $\gamma_i \stackrel{\text{iid}}\sim \mathcal N(0, \sigma^2_\gamma)$ is independent of $\e$ but I'll let $\Cov(\alpha, \gamma) = \rho\sigma_\gamma\sigma_\alpha I$. $\beta$ is the fixed global slope and $\gamma_i$ represents group-level deviations from that. $\rho$ is the correlation between $\alpha_i$ and $\gamma_i$.

On page 7 of the lme4 manual Bates et al. comment on how $\rho \neq 0$ can be used to make the model invariant to translations of $x$. When we see the form of the covariance we'll see why this is.

It can be shown that $$ \Cov(y_{ij}, y_{kl}) = \begin{cases} \sigma^2\delta_{jl} + \sigma^2_\alpha(1-\rho^2) + (\sigma_\gamma x_{ij} + \rho\sigma_\alpha)(\sigma_\gamma x_{kl} + \rho\sigma_\alpha) & i=j\\ 0 & i\neq j \end{cases} $$ where $\delta_{jl}$ is the Kronecker delta. I've written the covariance this way because it helps interpret what it means in terms of the covariance matrix to assume a random slopes model. It's the same as the random intercepts model except we also get this "similarity" term $(\sigma_\gamma x_{ij} + \rho\sigma_\alpha)(\sigma_\gamma x_{kl} + \rho\sigma_\alpha)$ which gives a large positive covariance when $x_{ij}$ and $x_{kl}$ are large with the same sign and a large negative covariance when they are large with different signs, and the larger the random slope variance $\sigma^2_\gamma$ is, the more the covariance structure is dominated by changes in the $x_{ij}$. Intuitively, when $\sigma^2_\gamma$ is relatively large, we expect to see $y_{ij}$ better explained by its slope term so we'll have a "clustering" around the group level trend lines.

This also shows what Bates et al. were referring to with $\rho$, in that if the $x_{ij}$ are all translated by a constant amount then a non-zero $\rho$ can account for this in the $\sigma_\gamma x_{ij} + \rho\sigma_\alpha$ terms. If $\rho = 0$ there's no way to "undo" the translation and then the estimates will actually change.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.