5
$\begingroup$

This is a question regarding the theory underlying mixed effects models, specifically a general rule of thumb that can be used to determine the structure of random-effect portion.

Here's what I understand:

(1) INCLUDE: RANDOM INTERCEPT: If you have more than one measurement for the variable (i.e. question is repeated over time or participant in a study answers multiple questions or survey administered over multiple time periods)

(2) INCLUDE: RANDOM INTERCEPT + RANDOM SLOPE: If you have more than one measurement for the FE-RE variable (i.e. participant in a study is exposed to more than one experimental condition)

How does one decide whether to make the intercept + slope relationship to be constrained or unconstrained?

Using the notation for LMER package in R, please see the difference below:

OPTION 1:

No constraint on the slope-intercept relationship. Just a random slope and random intercept:

lmer(Y ~ 1 + B + (1 + B | A), data=d)

OPTION 2:

Force B’s intercept and slope to be independent conditional on A then

lmer(Y ~ 1 + B + (1 | A) + (0 + B | A), data=d)

Credit: This site was wonderful in helping clarify the syntax. http://conjugateprior.org/2013/01/formulae-in-r-anova/

Unfortunately, I am not quite clear on the theory behind the choice identified above. When would it be statistically justified to pick Option 1 vs Option 2?

$\endgroup$
  • $\begingroup$ Thanks for the shout. Would running anova on the two models work to distinguish the specfications? $\endgroup$ – conjugateprior May 3 '16 at 1:37
  • $\begingroup$ Hi, so I think you could possibly get a model comparison using anova on the two models, no problem. However, I dont think an anova will test for the proper allocation of variance between Fixed and Random effects. Take for example the case where you do not include a slope for a random effect (random intercept model). It may increase the overall model fit, but artificially inflate the significance of a Fixed effect (i.e. inflate the likelihood of a Type I error). I wonder if it $\endgroup$ – HoneyBuddha May 3 '16 at 6:23
  • $\begingroup$ I think you could get a model comparison using anova on the 2 options, no prob. However, I dont think an anova will test for the "proper" allocation of variance between Fixed and Random effects. Take the example of a random intercept model. It may increase overall model fit, while also artificially inflating the sig. of a Fixed effect (i.e. inflate p(Type I err.). I wonder if there is a way to distinguish between these 2 options based upon the Experiment Design i.e. let the Exp. Design drive the modelling / struc of the Rand. Effects $\endgroup$ – HoneyBuddha May 3 '16 at 6:30
  • $\begingroup$ @conjugateprior, thanks! I followed your suggestion - Yes! An ANOVA can be used to distinguish the two. It appears your intuition was absolutely correct. I put what I found in my answer below. Check it out and let me know if that sounds about right to you. It's a delicate issue! One that has significant consequences for the kind of causal inferences you can draw from the model. Of course, I still cannot understand how to look at a design and determine if the slope-intercept should be constrained. And, if constrained, how should one translate that so that it can be expressed in broader terms. $\endgroup$ – HoneyBuddha May 3 '16 at 13:31
4
$\begingroup$

I'm just responding in case this might be useful for anyone else.


OPEN QUESTION

Turns out, I still have not found a clear resource that distinguishes between these two options in a theoretical manner - i.e. distinguishes between them not as an optimization but as being different model structures that are best suited for capturing different kinds of experimental designs.

Ideally, you should be able to rely upon your knowledge of the properties / structure of the study / data sample / data elicitation technique - and, make decisions about the random effects model before you have looked at the data.

After all, random effects should be determined a priori based on the theoretical assumptions.

That part of the question remains unanswered in a satisfactory manner

What are the theory-driven / experimental design based features that should allows us to determine whether Option 1 or Option 2 is the appropriate choice.


OPTION 1: random slope-intercept with no constraints

(G)LMER---|[ m.1 ]|---[Y ~ 1 + B + (1 + B | A)]|


OPTION 2: random slope-intercept with uncorrelated random effects

(G)LMER---|[ m.2 ]|---[Y ~ 1 + B + (1 | A) + (0 + B | A)]|



PARTIAL SOLUTION IN PRACTICE

However, I did find a a tutorial by Douglas Bates that may help. Around slide 73 onwards, he covers this topic. Essentially, this response is inspired by and often reproduces those slides. If you would like more detail, head there.

1. Inspect Your Random Effects Plots

Bates suggest that if visual inspection of the data plots gives you "little indication of a systematic relationship between a subject’s random effect for slope and his/her random effect for the intercept," we may want to consider using a model with uncorrelated random effects.

2. MODEL COMPARISON

2(a) Build Option 2 from above

First, we construct the model with the uncorrelated random effects. To express this we use two random-effects terms with the same grouping factor and different left-hand sides.

TWO GROUPING FACTORS:

  1. (1 | A)-----------[Random Intercept]
  2. (0 + B | A)-------[Random Slope, no intercept]
  3. Since the distinct random effects terms are modeled as being independent, by design, this imposes the constraint that the random intercept (1) from above is independent of the slope (2) conditional on A.

2(b) Compare the models using ANOVA

Using ANOVA for model comparison


  • Model m.1 represents the unconstrained random intercept-slope model associated with Option 1 from above

  • Model m.2 represents Option 2 where the intercept & slope are independent conditional on A


Model m.1 contains m.2 in the sense that:

If the parameter values for model m.1 were constrained so as to force the correlation (and, hence the covariance) to be zero and we could get the model to re-fit, we would get m.2


  1. Use a likelihood ratio test to determine if m.1 adds something substantial and statistically significant;

  2. If not, use the preference for parsimonious models (i.e. "smaller is better") and prefer the simpler, more constrained model;

  3. Since the value 0 to which the correlation is constrained is not on the boundary of the allowable parameter values, a likelihood ratio test and a reference distribution of a χ2 on 1 degree of freedom is suitable.

3. Likelihood ratio tests on variance component

As for the case of a covariance, we can fit the model with and without the variance component and compare the quality of the fits.

The likelihood ratio is a reasonable test statistic for the comparison but the “asymptotic” reference distribution of a χ2 does not apply because the parameter value being tested is on the boundary.

The p-value computed using the χ2 reference distribution should be conservative (i.e. greater than the p-value that would be obtained through simulation).

4. References and Resources

$\endgroup$
2
$\begingroup$

The following is a rule of thumb.

I think the default choice should be a model where the slope and the intercept covary ((1 + A | B)). Unless you have a theoretical or technical reason to exclude the possibility of covariance, it just makes in most cases a lot of sense to at least allow for the possibility. I wouldn't be able to come up with a theoretical reason, but a technical reason could be that you don't have enough data to estimate the covariance. On the other hand, even in such cases, estimating a covariance will likely improve the estimate of the random slopes and intercepts.

I'd be suspicious, or at least interested in finding out why it happens, if your crossvalidation or ANOVA or other "hard" measure (perhaps try WAIC?) is better for the model without covariance.

$\endgroup$
1
$\begingroup$

I think I may have an additional piece of information that can be used to help you. In the world of econometrics and more precisely, in the world of panel data estimation (the way econometritians call longitudinal/repeated measures data).

The test that I present to you is the Hausman specification test. The theoretical framework is to test, in terms of the estimator's consistency, if a random-effects model is preferred to a fixed-effects.

The approaches that I've used are more similar to your answer, but this can be one more idea for you.

Here are a few resources:

  1. Wikipedia article
  2. A nice video explaining the test
  3. Some slides on the test
  4. Chapter 10 on Woodridge's Econometric Analysis of Cross Section and Panel Data
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.