# Interpreting three forms of a “mixed model”

There's a distinction that's tripping me up with mixed models, and I'm wondering if I could get some clarity on it. Let's assume you've got a mixed model of count data. There's a variable you know you want as a fixed effect (A) and another variable for time (T), grouped by say a "Site" variable.

As I understand it:

glmer(counts ~ A + T, data=data, family="Poisson") is a fixed effects model.

glmer(counts ~ (A + T | Site), data=data, family="Poisson") is a random effect model.

My question is when you have something like:

glmer(counts ~ A + T + (T | Site), data=data, family="Poisson") what is T? Is it a random effect? A fixed effect? What's actually being accomplished by putting T in both places?

When should something only appear in the random effects section of the model formula?

-

## 3 Answers

This may become clearer by writing out the model formula for each of these three models. Let $Y_{ij}$ be the observation for person $i$ in site $j$ in each model and define $A_{ij}, T_{ij}$ analogously to refer to the variables in your model.

glmer(counts ~ A + T, data=data, family="Poisson") is the model

$$\log \big( E(Y_{ij}) \big) = \beta_0 + \beta_1 A_{ij} + \beta_2 T_{ij}$$

which is just an ordinary poisson regression model.

glmer(counts ~ (A + T|Site), data=data, family="Poisson") is the model

$$\log \big( E(Y_{ij}) \big) = \alpha_0 + \eta_{j0} + \eta_{j1} A_{ij} + \eta_{j2} T_{ij}$$

where $\eta_{j} = (\eta_{j0}, \eta_{j1}, \eta_{j2}) \sim N(0, \Sigma)$ are random effects that are shared by each observation made by individuals from site $j$. These random effects are allowed to be freely correlated (i.e. no restrictions are made on $\Sigma$) in the model you specified. To impose independence, you have to place them inside different brackets, e.g. (A-1|Site) + (T-1|Site) + (1|Site) would do it. This model assumes that $\log \big( E(Y_{ij}) \big)$ is $\alpha_0$ for all sites but each site has a random offset ($\eta_{j0}$) and has a random linear relationship with both $A_{ij}, T_{ij}$.

glmer(counts ~ A + T + (T|Site), data=data, family="Poisson") is the model

$$\log \big( E(Y_{ij}) \big) = (\theta_0 + \gamma_{j0}) + \theta_1 A_{ij} + (\theta_2 + \gamma_{j1}) T_{ij}$$

So now $\log \big( E(Y_{ij}) \big)$ has some "average" relationship with $A_{ij}, T_{ij}$, given by the fixed effects $\theta_0, \theta_1, \theta_2$ but that relationship is different for each site and those differences are captured by the random effects, $\gamma_{j0}, \gamma_{j1}, \gamma_{j2}$. That is, the baseline is random shifted and the slopes of the two variables are randomly shifted and everyone from the same site shares the same random shift.

what is T? Is it a random effect? A fixed effect? What's actually being accomplished by putting T in both places?

$T$ is one of your covariates. It is not a random effect - Site is a random effect. There is a fixed effect of $T$ that is different depending on the random effect conferred by Site - $\gamma_{j1}$ in the model above. What is accomplished by including this random effect is to allow for heterogeneity between sites in the relationship between $T$ and $\log \big( E(Y_{ij}) \big)$.

When should something only appear in the random effects section of the model formula?

This is a matter of what makes sense in the context of the application.

Regarding the intercept - you should keep the fixed intercept in there for a lot of reasons (see, e.g., here); re: the random intercept, $\gamma_{j0}$, this primarily acts to induce correlation between observations made at the same site. If it doesn't make sense for such correlation to exist, then the random effect should be excluded.

Regarding the random slopes, a model with only random slopes and no fixed slopes reflects a belief that, for each site, there is some relationship between $\log \big( E(Y_{ij}) \big)$ and your covariates for each site, but if you average those effects over all sites, then there is no relationship. For example, if you had a random slope in $T$ but no fixed slope, this would be like saying that time, on average, has no effect (e.g. no secular trends in the data) but each Site is heading in a random direction over time, which could make sense. Again, it depends on the application.

Note that you can fit the model with and without random effects to see if this is happening - you should see no effect in the fixed model but significant random effects in the subsequent model. I must caution you that decisions like this are often better made based on an understanding of the application rather than through model selection.

-
(+1): writing out the model formula for each model is indeed the best way to make R-notations more transparent; good job! –  ocram Oct 1 '12 at 13:53
@Macro One question on the equations above (thanks for them btw) - do they also have the usual error term in them? If so, what's that term's subscript? –  Fomite Oct 2 '12 at 8:55
Hi - one way to write a GLM is as a model for $E(Y_{ij}|X)$ (or a 'linked' version) as I've done here. There is no error term for the expected value, if the model is correctly specified. To answer your question, in GLMs we're specifying the distribution of $Y_{ij}|X$. The "leftover" randomness in a linear model is manifested by a normally distributed error term. But, in non-linear GLMs (e.g. poisson, logistic) there is randomness "built in" since knowing the rate of a poisson or a success prob of a bernoulli trial doesn't allow you to predict a realization without error. Hope this helps. –  Macro Oct 2 '12 at 14:34

You should note that T is none of your model's a random effects terms, but a fixed effect. Random effects are only those effects that appear after the | in a lmer formula!

A more thorough discussion of what this specification does you can find in this lmer faq question.

From this questions your model should give the following (for your fixed effect T):

• A global slope
• A random slopes term specifying the deviation from the overall slope for each level of Site
• The correlation between the random slopes.

And as said by @mark999 this indeed is a common specification. In repeated measures designs, you generally want to have random slopes and correlations for all repeated measures (within-subjects) factors.

See the following paper for some examples (which I tend to always cite here):

Judd, C. M., Westfall, J., & Kenny, D. A. (2012). Treating stimuli as a random factor in social psychology: A new and comprehensive solution to a pervasive but largely ignored problem. Journal of Personality and Social Psychology, 103(1), 54–69. doi:10.1037/a0028347

-
A similar reference from ecology: Schielzeth, Holger, and Wolfgang Forstmeier. 2009. “Conclusions Beyond Support: Overconfident Estimates in Mixed Models.” Behavioral Ecology 20 (2) (March 1): 416–420. doi:10.1093/beheco/arn145. beheco.oxfordjournals.org/content/20/2/416. –  Ben Bolker Oct 1 '12 at 18:45

Something should appear only in the random part when you are not particularly interested in its parameter, per se, but need to include it to avoid dependent data. E.g., if children are nested in classes, you usually want children only as a random effect.

-
Maybe I'm misunderstanding you, but I would have thought that having fixed and random effects for the same variable was more common than a variable having just a random effect. Having fixed and random effects for the same variable is not uncommon in the Pinheiro and Bates book. –  mark999 Oct 1 '12 at 11:16
@MichaelChernick as I understand it, if you have a fixed effect and a random effect for the same variable, then the fixed effect is the overall effect in the population, while the random effect allows a different effect of the variable for each subject. There are several examples in Pinheiro & Bates. –  mark999 Oct 1 '12 at 11:43
@PeterFlom, re: "if children are nested in classes, you usually want children only as a random effect." I think you mean that class is the random effect. Unless there is further nesting in the data (e.g. repeated measurements on kids) then child level random effects are not identified. –  Macro Oct 1 '12 at 13:17
@macro Yes, that's what I meant, sorry. The terminology gets very confusing! That may be why Gelman eschews the terms 'fixed' and 'random' –  Peter Flom Oct 1 '12 at 16:45
@Michael, I agree with you. In these kinds of hierarchical models, the random effects are defined by a grouping variable (as opposed to other multivariate models such as spatially indexed data sets, where the 'grouping' variable is continuously varying). In the OP's question, Site would be referred to as the random effect, not T or A or anything else. Thinking of it that way, Site's effect clearly could not be both fixed and random, since the two wouldn't be identified from each other. You can have both fixed and random coefficients for a variable, but that's a different question. –  Macro Oct 1 '12 at 17:23