# Generalized Linear Mixed Effects

When to use multivariate logistic regression versus generalized linear mixed-effects models? What is the difference between the two?

Edit (in response to comments):

I was hoping to see multivariate as more than one covariate.

My problemset is analysis for a longitudinal prospective cohort study where im measuring vitamin K intake in relation to incidence of calcification. vitamin K intake is primarily bein assessed by food questionnaires but im also looking at plasma vitamin K levels. the exposure variable and the outcome variable are ordinal/categorical. on the other hand, outcome also has more than two categories. i will be using a multivariate mixed model. considering i know my covariates, it should be suffice to use a mixed regression model? Thank you .

• can you pls be a bit more specific, multivariate logistic regression is a special case of GLMM. AFAIK. Mar 18, 2011 at 2:31
• What do you mean by 'multivariate' in 'multivariate logistic regression'? 'Multivariate, i.e. more than one outcome variable which you wish to include in the same model? Multivariable, i.e. logistic regression with more than one covariate? Multinomial, i.e. outcome has more than two categories? Mar 18, 2011 at 7:38
• Here are two review papers that I hope will help you sort out the terminology and possible applications for GLMM: Generalized Linear Mixed Models, Statistical inference in generalized linear mixed models: A review -- the second one is a bit more technical as it mainly deals with estimation methods.
– chl
Mar 29, 2011 at 6:30

## 1 Answer

This depends on what you mean by "multivariate". Doing a quick google search, the term seems to refer to "more than 1 co variate". So the model equation is for "ordinary logistic regression" you have:

$$log(\frac{\pi}{1-\pi})=\beta_0 + \beta_1 X_1$$

For "multivariate logistic regression" you have:

$$logit(\frac{\pi}{1-\pi})=\beta_0 + \beta_1 X_1 + \beta_2 X_2 +\dots+ \beta_p X_p=X\beta$$

Where $X\beta$ is the "matrix version" - just for more compact notation. But as @onestop points out, the term "multivariate" could also refer to more than 1 probability. The model equations are then:

$$logit(\frac{\pi_1}{\pi_m})=X^{(1)}\beta^{(1)}$$ $$logit(\frac{\pi_2}{\pi_m})=X^{(2)}\beta^{(2)}$$ $$\dots$$ $$logit(\frac{\pi_{m-1}}{\pi_m})=X^{(m-1)}\beta^{(m-1)}$$

Where the brackets $(1),(2),..,(m-1)$ just indicate that the co variates and betas could be different in each model. The "multinomial" simplifies back to the "multivariate" when $m=2$ (only 2 categories), and the "multivariate" simplifies back to the "ordinary logistic regression" when you have only 1 co variate.

Now these are all "independence" based models, in that given the covariates and betas, any one individual's probability tells you nothing about any others. When you move from this framework, which is called "generalised linear modelling" or GLM in the literature, to the "extra M" of GLMM, the observations are now assumed to somehow depend on each other. The easiest way I can see a reason for this happening is that you know a particular attribute will increase or decrease the probability of a certain group of observations, but you have not measured that covariate in your data (e.g. you are analyzing the incidence of cancer among people, but you do not know which people in your data set are smokers. However have variables which could predict smoking status.). The usual GLM model goes:

$$DATA=MODEL+NOISE^{(1)}$$

(NOTE: although the likelihood isn't linear for GLMs, the usual fisher-scoring IRLS or newton-rhapson algorithm means this is the effective model one is actually estimating - because it is based on a 1 term taylor series expansion "the delta method").

Whereas the GLMM goes:

$$DATA=MODEL+MODEL(UNOBSERVED)+NOISE^{(2)}$$

The unobserved model is sometimes written as $Z\gamma$. So your model equation becomes:

$$logit(\frac{\pi}{1-\pi})=X\beta+Z\gamma$$

The vector $\gamma$ is often called a "random effect" (and is usually assumed to have a normal distribution) and the vector $\beta$ is called a "fixed effect" (although I don't particularly like this terminology).

And the idea is to "decompose" the noise into the "correlated part" and the "independent part". You fit the "fixed effect" part of the model - this is like the "global model". The GLMM basically allows you to recognise that there can be "local effects" in your data - systematic deviations from the "global model" for small groups, in a very similar way to ANOVA and ANCOVA. In fact, all of the GLMMs I have fit, my $Z$ matrix has been a classical design matrix for ANOVA.

So basically what the GLMM does is make a compromise between the global model and the local model. The compromise depends on the estimated variance for the "random effect" and on the estimate variance of the noise.