1
$\begingroup$

I have a bit of a theoretical question about random effects models and regression. If I have a set of clustered, longitudinal data (say repeated measurements of $y$ on a number of different individuals) is there really any difference, aside for adjustments to degrees of freedom, perhaps, to fitting a regression model for each individual and averaging the parameter estimates across models to get an overall average model (i.e. all fixed effects) vs. fitting a model with individual id as a random effect in which each of the individual's observations fall (i.e a mixed model)?

Thanks.

| cite | improve this question | |
$\endgroup$
  • $\begingroup$ Are you assuming you have an identical number of points per individual, or that you are doing a weighted averaging afterwards? $\endgroup$ – mkt - Reinstate Monica Jul 21 '19 at 19:53
  • $\begingroup$ Incidentally, your question could also be reframed as: "What's the difference between including individual as a fixed effect vs. a random effect?". And I discuss this a bit in my answer here: stats.stackexchange.com/a/289346/121522 $\endgroup$ – mkt - Reinstate Monica Jul 21 '19 at 19:55
  • $\begingroup$ @hi, mkt, I should have been more clear that I'm assuming equal sizes or weighted averaging. Thanks for the link. I'll give it a read now. $\endgroup$ – StatCurious Jul 21 '19 at 20:42
  • $\begingroup$ See Gelman's the Secret Weapon. $\endgroup$ – kjetil b halvorsen♦ Nov 26 '19 at 2:41
2
$\begingroup$

Here's 3 cases to consider:

  1. Separate linear regressions per individual
  2. A single regression with a fixed effect for individual and an interaction term allowing individuals to have different slopes
  3. A mixed model with random individual-level intercepts and slopes

1 and 2 are equivalent in terms of their mean predictions, though you're correct that the degrees of freedom & standard errors will differ. Since you don't care about that, I'll treat them as identical for purposes of comparing them against 3.

3 is different because of how random effects work: the individual-level random effects are shrunk towards zero i.e. you will tend to get less extreme values than you did in 1 & 2. This shrinkage is an important reason why random effects are valuable; it's not just about saving degrees of freedom.

A good description of why this works is in this old article on Stein's paradox in statistics

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.