20
$\begingroup$

I've skimmed through several books (Raudenbush & Bryk, Snijders & Bosker, Gelman & Hill, etc.) and several articles (Gelman, Jusko, Primo & Jacobsmeier, etc.), and I still haven't really wrapped my head around the major differences between using clustered standard errors verses multilevel modeling.

I understand the parts that have to with the research question at hand; there are certain types of answers you can only get from multilevel modeling. However, for example, for a two-level model where your coefficients of interest are only at the second level, what is the advantage of doing one method over the other? In this case, I'm not worried about making predictions or extracting individual coefficients for clusters.

The main difference I've been able to find is that clustered standard errors suffer when clusters have unequal sample sizes and that multilevel modeling is weak in that it assumes a specification of the random coefficient distribution (whereas using clustered standard errors is model-free).

And in the end, does all of this mean that for models that could ostensibly use either method, we should be getting similar results in terms of coefficients and standard errors?

Any responses or helpful resources would be greatly appreciated.

$\endgroup$
4
  • 7
    $\begingroup$ user Stask has a nice answer to exactly this question. $\endgroup$ – Andy W Jun 3 '13 at 19:37
  • $\begingroup$ Thanks. I did read that before, which actually made me more skeptical of the real benefits. However, I guess the real motivation behind my question is to see whether I'm at all validated in thinking that it's not overly useful if I'm only looking at level-two coefficients as being of interest. In addition, perhaps I missed it, but I don't think that post addressed whether these two methods should be producing similar results (when assumptions of both methods are met). $\endgroup$ – RickyB Jun 4 '13 at 5:48
  • 1
    $\begingroup$ With "coefficients at the second level" you mean the level where the you the parameters of the first stage as dependent variables? $\endgroup$ – sheß Sep 2 '15 at 23:33
  • $\begingroup$ Yes, that is what I mean. $\endgroup$ – RickyB Sep 18 '15 at 18:19
1
$\begingroup$

This post bases on personal experiences which might be specific to my data, so I'm not sure it qualifies as an answer.

I suggest to use simulations if possible to assess which method works best for your data. I did this and was surprised to find that tests (regarding parameters in the first level) based on multilevel modelling were outperforming any other method (power-wise), while retaining size even in small samples with few and unevenly sized "clusters". I am yet to find a paper that makes that point, and from how I see this is not really a niche topic and deserves more attention. I think it is fairly under-researched how different methods compare vis-a-vis finite-sample or few/uneven clusters.

$\endgroup$
1
  • $\begingroup$ Thank you for your comment. Do you happen to have any document where you recorded your results? I would be very interested in looking at it and seeing what you found (and, of course, I would not cite, share, or improve on it without discussing it with you). $\endgroup$ – RickyB Sep 18 '15 at 18:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.