3
$\begingroup$

I find the available (online, freely available) literature on linear mixed models to be comprehensive but utterly convoluted.

Assume we have some longitudinal data. We have a dependent variable, $Y_{it}$, that is indexed by time and a group. Within a group, observations over time would have some level of autocorrelation (i.e. the independence of observations within a group doesn't hold up).

There are also independent variables, $X$, which could be indexed by time, $X_{t}$, or by both group and time, $X_{it}$, depending on how the model is set up. Furthermore, one would expect that the intercept and slope would vary across each group.

As far as I can tell, this appears to be a case for a linear mixed model. How best would one construct a suitable model formula for the above scenario?


To provide more context, let's imagine we are trying to estimate some measure of the economic cycle, $Z_{it}$ for different countries $i$ and time $t$.

To predict this $Z_{it}$, we have two predictors. Let's imagine these are global GDP and the unemployment rate for each country $i$. The measure of GDP will vary with time $t$ but not with country $i$ because it's not country specific. The measure of unemployment will vary with both country and time.

For further clarity, let's imagine the data is arranged something like this

$$\begin{array}{|c|c|c|c|c|} \hline \text{Z} & \text{Country} & \text{Date} & \text{GDP} & \text{Unemployment}\\ \hline -2.4 & A & \text{Jan } 2010 & 0.3 & 4.8\\ 2.8 & A & \text{Feb } 2010 & 0.4 & 4.9\\ 1.3 & A & \text{Mar } 2010 & 0.7 & 5.2\\ 3.1 & A & \text{Apr } 2010 & 0.8 & 4.9\\ \vdots & \vdots & \vdots & \vdots & \vdots \\ 3.3 & B & \text{Jan } 2010 & 0.3 & 3.2\\ 1.5 & B & \text{Feb } 2010 & 0.4 & 3.1\\ -2.3 & B & \text{Mar } 2010 & 0.7 & 2.9\\ 4.0 & B & \text{Apr } 2010 & 0.8 & 2.8\\ \vdots & \vdots & \vdots & \vdots & \vdots \\ \hline \end{array}$$

So we can see that the GDP variable is the same for each country at a given date.

For a given country, the $Z_{it}$ would be autocorrelated because we would expect that $Z_{i(t+1)}$ to be correlated with $Z_{it}$. As far as I understand, the country variable would be used to group observations. The GDP and unemployment variables would enter as dynamic predictors.

As such, a model set up like

lmer(Z ~ Country + GDP + Unemployment + (1 | Date), data = data)

would give a random effect for the date and fixed effects for the country and economic variables. Does this seem appropriate?

Moreover, to round out the question, what are the advantages of using this mixed model as opposed to using a simple linear model

lm(Z ~ Country + GDP + Unemployment, data = data)

in this instance?

$\endgroup$
3
$\begingroup$

When the online references become convoluted, it is time to invest in some good books on the topics of mixed effects modelling or longitudinal data analysis. You can pose a question on this forum asking people to recommend such books to you.

Going back to your question, the first thing you should note is that your terminology needs to be more nuanced. In the context of mixed effects modelling, there are such things as:

  1. Random grouping factors;
  2. Study groups or experimental groups.

Scenario 1: Say you have a study/experiment where you randomly select some individuals from a target population of interest to you and measure the value of a response variable (e.g., weight in kg) for each individual at multiple points in time (e.g., monthly for a period of 3 months). In this setting, the individual can be viewed as a random grouping factor since it groups together multiple response values coming from the same individual, which are expected to correlated as they are all affected by the same external or internal factors which affect that individual.

Scenario 2: Say you have a study where you select some eligible individuals from a target population of interest to you and then you randomly assign them to a new treatment (Group A) or a standard treatment (Group B). After this assignment, you record the values of a response variable of interest (e.g., weight in kg) for each individual at multiple time points (e.g., once a month for a period of 3 months). In this setting, the individual can still be viewed as a random grouping factor; the Group (A or B) can be viewed as a study group.

Both of these scenarios will lead to data which will have an hierarchical structure. On the top level of the data hierarchy, you have the individuals. On the bottom level of the hierarchy, you have the repeated values of the response variable measured for each individual. Something like this:

  😊            😊            😊           😊        😊       

71,71,72      68,67,69      73,70,71     65,65,67   81,82,82

As you pointed out, when you have an hierarchical data structure like this, where the response values are nested within the individual, you can have other predictor variables to consider. These predictor variables can be measured at either the top level of the hierarchy (subject-level) or at the bottom level of the hierarchy (response-level, aka observation-level).

An example of subject-level predictor would be gender. This predictor would be considered a static predictor since its values would not be expected to change across response times for the same individual.

Another example of subject-level predictor would be Group (A or B) for the second scenario. This predictor would also be considered a static predictor since its values would not change across response times for the same individual. However, there are studies/experiments where a subject would take turns participating in each group. If that is the case, Group would be treated as a dynamic predictor measured at the response-level of the data hierarchy.

An example of response-level predictor is blood pressure (if it is measured at the same time with weight and its values are expected to change across response times). A response-level predictor is dynamic.

Another example of response-level predictor is time, which is obviously dynamic.

If you need to formulate a linear mixed effects model for your 2-level data hierarchy, you have to be clear upfront about what predictors you will include in your model and whether they are static or dynamic.

From your post, I cannot tell whether you are interested in learning more about the mathematical or software formulation of linear mixed effects models.

If it is the software formulation, here is what models for Scenario 1 could look like in lmer formulation if your choice of software is R:

library(lme4)

m1 <- lmer(weight ~ time + (1|subject), data = yourdata)

m2 <- lmer(weight ~ time + (1 + time|subject), data = yourdata)

The first model, m1, postulates that weight increases at the same linear rate for each individual but allows individuals to start with different weights at the beginning of the study, if time is coded as 0 for month 1, 1 for month 2 and 2 for month 3. This model is typically referred to as a random intercept model.

The second model, m2, postulates that weight increases at different linear rates for each individual while also allowing individuals to start with different weights at the beginning of the study, presuming time is coded as 0 for month 1, 1 for month 2 and 2 for month 3. This model is typically referred to as a random intercept and random slope model. The model also assumes that there is a correlation between the random intercepts and the random slopes corresponding to the individuals included in the model (e.g., individuals which start out with higher weights tend to experience larger rates of weight gain).

If this last assumption is not tenable for your data, you can impose no correlation between the random intercepts and random slopes:

m2 <- lmer(weight ~ time + (1 + time||subject), data = yourdata)

For Scenario 1, if you have another dynamic predictor such as blood pressure (bp), you can include it in, say, model m2, either with the same slope across individuals or with different slopes across individuals:

m2a <- lmer(weight ~ time + bp + (1 + time|subject), data = yourdata)

m2b <- lmer(weight ~ time + bp + (1 + time + bp|subject), data = yourdata)

You can also allow an interaction between time and bp and assume the interaction effect to be the same across individuals or different across individuals:

m2c <- lmer(weight ~ time + bp + time:bp + (1 + time|subject), data = yourdata)

m2d <- lmer(weight ~ time + bp + time:bp + (1 + time + bp + time:bp|subject), data = yourdata)

The bottom line here is that only dynamic predictors can have slopes that vary across individuals.

For Scenario 2, where time is a dynamic predictor and group is a static predictor, possible models include:

M1 <- lmer(weight ~ time + group + (1|subject), data = yourdata)

M2 <- lmer(weight ~ time + group + (1 + time|subject), data = yourdata)

M3 <- lmer(weight ~ time + group + time:group + (1|subject), data = yourdata)

M4 <- lmer(weight ~ time + group + time:group + (1 + time|subject), data = yourdata)

M5 <- lmer(weight ~ time + group + time:group + (1 + time + time:group|subject), data = yourdata)

If you interact a dynamic predictor with a static predictor, you get a dynamic predictor whose effect can vary across subjects.

Addendum

If you formulate your model like this:

lmer(Z ~ Country + GDP + Unemployment + (1 | Date), data = data)

you are essentially implying that you have multiple values of the response variable Z within at least some of your Dates (e.g., you measured Z both in the morning and in the afternoon for those Dates). This model formulation then allows for the possibility that values of Z collected within the same Date are correlated. From the example data you shared, this does not seem to be the case you are operating in - you only have one value of Z available per Date. Is that correct?

How you formulate your model depends on a number of things, including your research questions, the number of countries you have in your data and how those countries were selected for inclusion in your study.

If you only have a small number of countries to work with (say, less than 5), you might have no choice but to consider an lm() model for your countries. Even so, your model might need to allow for the possibility of correlated errors over time within each country. If there is indication of temporal correlation of the lm() model residuals, you would need to switch to using the gls() function from the nlme package of R, which can handle temporal correlation among model errors. These considerations would apply whether you selected the small number of countries so as to be representative of a larger set of countries you are truly interested in or if you chose those countries because they were the only ones you were interested in.

If you have more countries to work with (say, 5 or more) and those countries were selected for inclusion in your study because they are representative of a larger set of countries you are truly interested in, then you could consider a linear mixed effects model as implemented by the lmer() function in the lme4 package of R. The model would treat Country as a random grouping factor so that the multiple values of Z collected on different Dates for a country are allowed to be correlated over time.

lmer(Z ~  GDP + Unemployment + (1 | Country), data = data)

If you consider Country as a random grouping factor (i.e., Country is like a smiley face 😊), then Date can be treated as a dynamic predictor measured at the lowest level of your data hierarchy, so in principle you could include it in your model if you wanted to adjust the effects of GDP and Unemployment for the effect of time. Whether you include Date in your model depends on your research question. How you include it in the model depends on whether you believe the effect of time to be linear or nonlinear. For a linear effect, you would use something like this:

lmer(Z ~  GDP + Unemployment + Date + (1 | Country), data = data)

So the answer to your last question is deeper than just rattling the advantages and disadvantages of lm() versus lmer() - you really need to think about what it is you are trying to do and then do something that reflects the realities of your data and research questions.

$\endgroup$
2
  • 1
    $\begingroup$ Thanks for your very thorough response. I've made an edit to the question with some further context if that helps. $\endgroup$ – epp Oct 25 '20 at 6:17
  • $\begingroup$ @epp: See my Addendum for more insights. Also, I highly recommend investing in this excellent book: Longitudinal Data Analysis for the Behavioral Sciences Using R by Jeffrey D. Long. $\endgroup$ – Isabella Ghement Oct 25 '20 at 16:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.