5
$\begingroup$

I was taught that a Hausman test should be used in multilevel modeling in order to check whether random effects can be used.

However, I have now stumbled over several sources stating that the Hausman test is actually being misunderstood when using it in this way and that it rather tests whether the between and within effects are different. e.g.:

Snijders, T. A., & Bosker, R. J. (2011). Multilevel analysis: An introduction to basic and advanced multilevel modeling. Sage.

Bell, A., Fairbrother, M., & Jones, K. (2019). Fixed and random effects models: making an informed choice. Quality & Quantity, 53(2), 1051-1074.

The authors are referring to an article from 2004:

Fielding, A. (2004). The role of the Hausman test and whether higher level effects should be treated as random or fixed. Multilevel Model. Newsl. 16(2), 3–9.

There are, though, relatively recent textbooks treating the Hausman test as a tool to decide between fixed and random effects, e.g.:

Wooldridge, J. M. (2010). Econometric analysis of cross section and panel data. MIT press.

Could somebody please help me out of my confusion?

$\endgroup$
3
$\begingroup$

What the the Hausman test is doing is testing whether the results (i.e. the estimated coefficients) from a fixed effects and random effects model are significantly different.

(I haven't ever seen people talk about it testing whether whether the fixed and "between" effects are different, although because the random effects model is estimated as a weighted average of the between and fixed effects models, you could look at it that way, I suppose.)

However, the reason it's used to "decide between fixed vs random effects" is that if the results of the two models ARE different then that's a reason to prefer the fixed effects model.

Here's why.

A fixed effects (FE) model accounts for ALL omitted variable bias from variables at the higher "group" level, because a fixed effects model is basically just including a dummy indicator variable for each "group." This means that if you run a FE model you don't have to worry about omitted variable bias at the group level.

A random effects (RE) model does NOT automatically account for all group level bias (that is, bias due to differences "between groups"), although it allows you to include group level predictors (that is, predictor variables that vary ONLY between groups, but not within groups) that do that. Now, if you were to include ALL significant group level covariates in your random effects model then you would get the same answers (in terms of lower level coefficients) as the FE model (or rather, the two answers will approach each other as sample sizes goes to infinity). But the standard errors of the RE model will be smaller.

What all that means is that if the coefficients of a FE and RE model are basically the same, you should prefer the RE ones, because they have smaller standard errors, but if they are different you should prefer the FE ones because the fact that they are different suggests that there is omitted variable bias at the higher level that the RE model has not accounted for (but the FE model has). So that's why we use a Hausman test to see if the coefficients are different, and if the test says they are, we default to the FE model.

Now, I'll note that in "real life" there are many other criteria by which we might choose between these two models. Sometimes the independent variables we care about are at the higher level in which case we CAN'T use a FE model, since in a FE model you can't include any group-level predictors (because they are colinear with the dummy variables). So a Hausman test is only appropriate when you have other substantive reason to prefer one model over the other.

And I'll also note that in my experience the Hausman test is almost always significant: that is, it almost always tells you that the two models are significantly different even when that difference is so small as to be substantively meaningless. So, as is always the case in statistics: don't just blindly follow the results of the test.

$\endgroup$
3
  • $\begingroup$ Great answer. Maybe you could express more clearly that by group level variable you mean a variable which is constant per group (a group being a unit of observation). $\endgroup$
    – Helix123
    Dec 30 '20 at 19:28
  • $\begingroup$ In statistical parlance, if the test is conservative, it tends not to reject the null hypothesis. I suppose the null hypothesis is "no difference". If the test tends to reject that, it must be the opposite of conservative (liberal, I guess). Or did I get it wrong? $\endgroup$ Dec 30 '20 at 19:51
  • $\begingroup$ You're right, I shouldn't have used "conservative," (in my mind it's "conservative" because a significant tests suggests that there is still bias at the group level). I've edited the answer to avoid using that word. $\endgroup$ Dec 31 '20 at 13:19
3
$\begingroup$

Let's say the Hausman test favors the fixed effect model. Why might you still be interested in using the random effects model? The random effects or multilevel model allows a degree of flexibility in modeling that is much messier and in some cases impossible to implement in the fixed effect model. Examples include:

  1. Random slopes for the association between lower level variables and the outcome, which allow you to investigate whether the within-group association varies across groups. A classic example is growth curve modeling whereby the linear (or quadratic) rate of change in the outcome, codified by a continuous time variable) is allowed to vary across individuals or groups. In the fixed effect model, you can get at this in a crude way by interacting the lower-level variable with the cluster dummy variables. If you have a lot of groups, you have a lot of interactions.
  2. Related to the above, random effect models allow for interactions between within- and between-level predictors, which are sometimes called cross-level interactions. To investigate these in a random effects models requires three parameters - a random slope for the lower-level predictor, the covariance between the random slope and random intercept, and the fixed effect term for the interaction. This is not possible to investigate in the fixed effects modeling framework because all cluster-level variables are thrown out.
  3. Investigation of different within- and between- associations for lower-level variables. Here the interest is investigating whether a within-cluster association is different than the between-cluster association. This is sometimes termed contextual effects modeling and also allows you to identify cases of Simpson's paradox. An example is the association between exercise and having a heart attack. While exercising, one is at higher risk for a heart attack, but on average, individuals who exercise a lot tend to have lower risk for a heart attack. Relatedly, adding in the cluster means of lower-level variables takes care of the endogeneity problem at level 1 often used as a reason to prefer fixed effects models.
  4. Prediction - random effects models employ empirical Bayes prediction of the random intercepts and slopes. This is great because the procedure corrects for the reliability of the group's prediction. Smaller groups get pulled toward the sample average prediction.
$\endgroup$
3
  • $\begingroup$ How does this address the OP? You are not even mentioning the Hausman test. $\endgroup$ Dec 31 '20 at 8:30
  • 2
    $\begingroup$ I think the point being made (which I very much agree with) is that there are lots of reasons you might want to use a random effects model even when the hausman test is significant. Maybe that could be made more explicit though. $\endgroup$ Dec 31 '20 at 13:21
  • $\begingroup$ @GrahamWright Good point. I edited the answer to address why one might still prefer the random effects model even when the Hausman, a rather blunt test as you point out, suggest fixed effects are warranted. $\endgroup$
    – Erik Ruzek
    Dec 31 '20 at 17:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.