Real motivation for using mixed effect models, and when to use them and when not to

Question

My question might sound naïve, but despite my internet search, I wasn't able to find a satisfactory answer.

I've been introduced to linear regression, linear fixed effect and linear mixed effect models, mostly from part of the book by Fitzmaurice-Laird-Waire. In the opening chapters of the linear mixed models, they say that the principal reason for using the mixed effect model is that part of the regression parameters could vary from one individual (=subject) to another. I understand that partially (I guess!): so if we want to model the population say as $y(t)=a+bt$, then the initial value of the population, $a$ and the growth $b$ will depend on each individual. So it makes sense to consider $a,b$ as random variables rather than a constant, just like $y$, defined on the same sample space as $y$. But then I guess there'll be factors that influences growth, which will be the same for all subjects.

My question is: if some factor varies with subjects $S_i$ (say the growth rate $b_i$), why not use the $b_i$ as a constant in the model? What advantage will one gain assuming that $b_i$'s are random variables, as opposed to constants which are function of the subject $i$?

And, given a real life situation, how will one decide what model to use, fixed effect, or mixed effect, or purely random effect? Could you give me some examples where one will prefer fixed effect, and where mixed effect?

P.S. If this matters, I come from advanced pure mathematics background, and for me, fixed effect is just a special case of mixed effect, putting all $\vec{b_i}=\vec{0}$. But that's not how statisticians will view things I guess, because it's all a matter of which one is easier to use.

Answer 1

There are many reasons for using mixed or random effects models, but I'll highlight one due to my time constraints. Let's say you have 1500 subjects with 10 measurements taken on each subject, along with many covariates. You could model the response measurements $Y$ using fixed subject terms as $Y=X\beta+\epsilon.$ However, this would require entering 1499 dummy variable terms for subjects into the model (or 1500 if you didn't mind the $X$ matrix not being full rank), along with their covariates. Instead of using a fixed-effects approach for this, you could simply assume subject is a random effect with a given covariance structure (e.g. a compound symmetric covariance). You could then fit a fixed and random effects (called mixed effects) model as: $Y=X\beta+Zu+\epsilon.$ Using this approach, you'd only need to estimate the effects for the intercept (if any), for each of the covariates ($X$) in the model, and only two random effects for subjects ($\sigma_\epsilon^2$ and $\sigma_2$.) You no longer need to estimate separate effects for all those subjects!

A good way to determine if your factors should be random or fixed comes from Kleinbaum, et al.'s Applied Regression Analysis and Other Multivariable Methods book. It states the following:

"Fixed Factor: A variable in a regression model whose possible values (i.e. levels) are the only ones of interest. Random factor: A variable in a regression model whose levels are regarded as a random sample from some large population of levels."

It goes on to say:

"When applying the above definitions to epidemiologic studies, we typically postulate that:

a. Subjects, litters, observers, families, and households are random factors; b. Gender, age, marital status, day of the week, and education are fixed factors; and c. Locations, treatments, clinics, exposures, and time may be considered as either random or fixed factors, depending on the context of the study."

Further, it states:

"When in doubt, one approach for deciding how to classify a particular study variable is to consider the following question: 'If I was able to replicate the study, would I want a given factor to have the exact same categories as observed in the current study?' Equivalently, 'Would I want a replicate study to use the same treatments, days of week, or subjects as used int he current study?' If your answer is yes: treat the factor as fixed. no: treat the factor as random."