20
$\begingroup$

I've skimmed through several books (Raudenbush & Bryk, Snijders & Bosker, Gelman & Hill, etc.) and several articles (Gelman, Jusko, Primo & Jacobsmeier, etc.), and I still haven't really wrapped my head around the major differences between using clustered standard errors verses multilevel modeling.

I understand the parts that have to with the research question at hand; there are certain types of answers you can only get from multilevel modeling. However, for example, for a two-level model where your coefficients of interest are only at the second level, what is the advantage of doing one method over the other? In this case, I'm not worried about making predictions or extracting individual coefficients for clusters.

The main difference I've been able to find is that clustered standard errors suffer when clusters have unequal sample sizes and that multilevel modeling is weak in that it assumes a specification of the random coefficient distribution (whereas using clustered standard errors is model-free).

And in the end, does all of this mean that for models that could ostensibly use either method, we should be getting similar results in terms of coefficients and standard errors?

Any responses or helpful resources would be greatly appreciated.

Improve this question
$\endgroup$
4
  • 7
    $\begingroup$ user Stask has a nice answer to exactly this question. $\endgroup$
    – Andy W
    Jun 3, 2013 at 19:37
  • $\begingroup$ Thanks. I did read that before, which actually made me more skeptical of the real benefits. However, I guess the real motivation behind my question is to see whether I'm at all validated in thinking that it's not overly useful if I'm only looking at level-two coefficients as being of interest. In addition, perhaps I missed it, but I don't think that post addressed whether these two methods should be producing similar results (when assumptions of both methods are met). $\endgroup$
    – RickyB
    Jun 4, 2013 at 5:48
  • 1
    $\begingroup$ With "coefficients at the second level" you mean the level where the you the parameters of the first stage as dependent variables? $\endgroup$
    – sheß
    Sep 2, 2015 at 23:33
  • $\begingroup$ Yes, that is what I mean. $\endgroup$
    – RickyB
    Sep 18, 2015 at 18:19

1 Answer 1

Reset to default
1
$\begingroup$

This post bases on personal experiences which might be specific to my data, so I'm not sure it qualifies as an answer.

I suggest to use simulations if possible to assess which method works best for your data. I did this and was surprised to find that tests (regarding parameters in the first level) based on multilevel modelling were outperforming any other method (power-wise), while retaining size even in small samples with few and unevenly sized "clusters". I am yet to find a paper that makes that point, and from how I see this is not really a niche topic and deserves more attention. I think it is fairly under-researched how different methods compare vis-a-vis finite-sample or few/uneven clusters.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you for your comment. Do you happen to have any document where you recorded your results? I would be very interested in looking at it and seeing what you found (and, of course, I would not cite, share, or improve on it without discussing it with you). $\endgroup$
    – RickyB
    Sep 18, 2015 at 18:18
  • 1
    $\begingroup$ This paper wasn't published at the time of this answer, but for anyone who's still curious about this question, McNeish, Stapleton, & Silverman (2016) address this and cite a few papers that discuss the issues with imbalanced clusters noted in the answer above. Full text here: researchgate.net/publication/… $\endgroup$
    – EmKayDee
    Apr 24 at 15:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.