As explained in these introduction to random intercepts models lesson, for a simple random intercepts model, the model definition is the following: $$y_{ij} = b_0+b_1x_{ij}+u_j+e_{ij}$$ Where $y_{ij}$ is the response of the individual $i$ which is in group $j$, $b_0$ is the intercept, $b_1$ the slope, $x_{ij}$ the predictor for that individual, $u_j$ the random intercept corresponding to group $j$ and $e_{ij}$ a random noise.

My question is: is this model equivalent to a linear model with more variables, $x$ and a dummy variable for every group? And more generally, can every mixed model be transformed into a generalized linear model?

  • $\begingroup$ It will depend on the distribution you specify for the error terms $e_{ij}$. $\endgroup$ – Dimitris Rizopoulos Jun 14 '19 at 10:36
  • $\begingroup$ If they are $N(0, \sigma^2)$, is it the same though? In which case are they different? $\endgroup$ – David Masip Jun 14 '19 at 11:03
  • 1
    $\begingroup$ You could for example assume that the error terms have a t distribution. More specifically, this is not just a linear regression with more variables. It’s a linear mixed model. The random effect $u$ is an unobserved variable. When you define the model for the observed data $y$ alone, integrating over $u$, you get a multivariate regression model with correlated error terms. $\endgroup$ – Dimitris Rizopoulos Jun 14 '19 at 21:40

This is not a generalized linear model (well, technically it is), it is just a plain old linear model!

You can define a dummy variable for each of the groups (except one "reference group") You can then build your individuals-variables matrix as in any linear model, and solve by least-squares to estimate the parameters.

EDIT: I am not sure about what exactly counts as a "mixed model". Many of those names are used for historical reasons, as the general formulation of the linear model has only been around for about half a century (It is confusing, but please don't mistake "general linear model" with "generalized linear model". The former refers to any model than can be written as $Y=AX+\epsilon$ while the latter also allows for some transformations that make it suitable for other problems like classification (logistic regression is probably the most famous case of a generalized linear model)

Another reason why those specific names exist is clarity in description, as, although mathematically the model you describe is just "multivariant regression", from a "functional" point of view, it's quite different (those "dummy variables" are just a trick we use to make it fit into the linear model form, but we don't often think of cateogrical variables as their dummy representation)

Finally, I guess pretending we are working with a set of different models, rather than a general one, makes everyone else think we are smarter!

  • $\begingroup$ So they are the same thing in this case, right? $\endgroup$ – David Masip Jun 14 '19 at 10:19
  • $\begingroup$ Yes. What you have in your hands is, from an algrebraic perspective, no different that multiple linear regression $\endgroup$ – David Jun 14 '19 at 10:23
  • $\begingroup$ And is this true for any mixed model? And if so, where is the need to provide cool names such as random intercepts while they are doing a linear model? $\endgroup$ – David Masip Jun 14 '19 at 10:25
  • $\begingroup$ @DavidMasip The answer may be a bit complex, so I'll edit my original response $\endgroup$ – David Jun 14 '19 at 10:32
  • 1
    $\begingroup$ Part of this answer is not actually correct; the $u_j$ in a random intercept model can't be estimated using dummy variables (that would be a fixed-effects model.) The link in the OP makes this clear. However, you are correct that it is not a generalized linear model. $\endgroup$ – jbowman Jun 14 '19 at 14:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.