Under what conditions should one use multilevel/hierarchical analysis?

Question

Under which conditions should someone consider using multilevel/hierarchical analysis as opposed to more basic/traditional analyses (e.g., ANOVA, OLS regression, etc.)? Are there any situations in which this could be considered mandatory? Are there situations in which using multilevel/hierarchical analysis is inappropriate? Finally, what are some good resources for beginners to learn multilevel/hierarchical analysis?

Answer 1

When the structure of your data is naturally hierarchical or nested, multilevel modeling is a good candidate. More generally, it's one method to model interactions.

A natural example is when your data is from an organized structure such as country, state, districts, where you want to examine effects at those levels. Another example where you can fit such a structure is is longitudinal analysis, where you have repeated measurements from many subjects over time (e.g. some biological response to a drug dose). One level of your model assumes a group mean response for all subjects over time. Another level of your model then allows for perturbations (random effects) from the group mean, to model individual differences.

A popular and good book to start with is Gelman's Data Analysis Using Regression and Multilevel/Hierachical Models.

Answer 2

The Centre for Multilevel Modelling has some good free online tutorials for multi-level modeling, and they have software tutorials for fitting models in both their MLwiN software and STATA.

Take this as heresy, because I have not read more than a chapter in the book, but Hierarchical linear models: applications and data analysis methods By Stephen W. Raudenbush, Anthony S. Bryk comes highly recommended. I also swore there was a book on multi level modeling using R software in the Springer Use R! series, but I can't seem to find it at the moment (I thought it was written by the same people who wrote the A Beginner’s Guide to R book).

edit: The book on using R for multi-level models is Mixed Effects Models and Extensions in Ecology with R by Zuur, A.F., Ieno, E.N., Walker, N., Saveliev, A.A., Smith, G.M.

good luck

Answer 3

Here's another perspective on using multilevel vs. regression models: In an interesting paper by Afshartous and de Leeuw, they show that if the purpose of the modeling is predictive (that is, to predict new observations), the choice of model is different from when the goal is inference (where you try to match the model with the data structure). The paper that I am referring to is

Afshartous, D., de Leeuw, J. (2005). Prediction in multilevel models. J. Educat. Behav. Statist. 30(2):109–139.

I just found another related paper by these authors here: http://moya.bus.miami.edu/~dafshartous/Afshartous_CIS.pdf

Answer 4

Here's an example where a multilevel model might be "essential." Suppose you want to rate the "quality" of the education provided by a set of schools using students' test scores. One way to define school quality is in terms of average test performance after taking student characteristics into account. You could conceptualized this as, $$y_{is} = \alpha_s + X_{is}'\beta_s + \epsilon_{is},$$ where $y_{is}$ is the continuous test score for student $i$ in school $s$, $X_{is}$ are student attributes centered at school means, $\beta_s$ is a school-specific coefficient on these attributes, $\alpha_s$ is a "school effect" that measures school quality, and $\epsilon_{is}$ are student level idiosyncrasies in test taking performance. Interest here focuses on estimating the $\alpha_s$'s, which measure the "added value" that the school provides to students once their attributes are accounted-for. You want to take student attributes into account, because you don't want to punish a good school that has to deal with students with certain disadvantages, therefore depressing average test scores despited the high "added value" that the school provides to its students.

With the model in hand, the issue becomes one of estimation. If you have lots of schools and lots of data for each school, the nice properties of OLS (see Angrist and Pischke, Mostly Harmless..., for a current review) suggest that you would want to use that, with suitable adjustments to standard errors to account for dependencies, and using dummy variables and interactions to get at school level effects and school specific intercepts. OLS may be inefficient, but it's so transparent that it might be easier to convince skeptical audiences if you use that. But if your data are sparse in certain ways---particularly if you have few observations for some schools---you may want to impose more "structure" on the problem. You may want to "borrow strength" from the larger-sample schools to improve the noisy estimates that you would get in the small-sample schools if the estimation were done with no structure. Then, you might turn to a random effects model estimated via FGLS, or maybe an approximation to direct likelihood given a certain parametric model, or even Bayes on a parametric model.

In this example, the use of a multilevel model (however we decide to fit it, ultimately) is motivated by the direct interest in the school-level intercepts. Of course, in other situations, these group level parameters may be nothing more than nuisance. Whether or not you need to adjust for them (and, therefore, still work with some kind of multilevel model) depends on whether certain conditional exogeneity assumptions hold. On that, I would recommend consulting the econometric literature on panel data methods; most insights from there carry over to general grouped data contexts.

Answer 5

Multi-level modelling is appropriate, as the name suggests, when your data have influences occurring at different levels (individual, over time, over domains, etc). Single level modeling assumes everything is occurring at the lowest level. Another thing that a multi-level model does is to introduce correlations among nested units. So level-1 units within the same level-2 unit will be correlated.

In some sense you can think of multi-level modelling as finding the middle ground between the "individualist fallacy" and the "ecological fallacy". Individualist fallacy is when "community effects" are ignored such as the compatibility of a teacher's style with a student's learning style, for example (the effect is assumed to come from the individual alone, so just do regression at level 1). whereas "ecological fallacy" is the opposite, and would be like supposing the best teacher had the students with the best grades (and so that the level-1 is not needed, just do regression entirely at level 2). In most settings, neither is appropriate (the student-teacher is a "classical" example).

Note that in the school example, there was a "natural" clustering or structure in the data. But this is not an essential feature of multi-level/hierachical modeling. However, the natural clustering makes the mathematics and computations easier. The key ingredient is the prior information which says that there are processes happening at different levels. In fact you can devise clustering algorithms by imposing a multi-level structure on your data with uncertainty about which unit is in which higher level. So you have $y_{ij}$ with the subscript $j$ being unknown.

Answer 6

Generally, speaking a hierarchical bayesian (HB) analysis will lead to efficient and stable individual level estimates unless your data is such that individual level effects are completely homogeneous (an unrealistic scenario). The efficiency and stable parameter estimates of HB models becomes really important when you have sparse data (e.g., less no of obs than the no of parameters at the individual level) and when you want to estimate individual level estimates.

However, HB models are not always easy to estimate. Therefore, while HB analysis usually trumps non-HB analysis you have to weigh the relative costs vs benefits based on your past experience and your current priorities in terms of time and cost.

Having said that if you are not interested in individual level estimates then you can simply estimate an aggregate level model but even in these contexts estimating aggregating models via HB using individual level estimates may make a lot of sense.

In summary, fitting HB models is the recommended approach as long as you have the time and the patience to fit them. You can then use aggregate models as a benchmark to assess the performance of your HB model.

Answer 7

I learned from Snijders and Bosker, Multilevel Analysis: An introduction to basic and advanced multilevel modeling. It is very well pitched at the beginner I think, it must be because I am a thicko where these things are concerned and it made sense to me.

I second the Gelman and Hill as well, a truly brilliant book.

Answer 8

Multi-level models should be employed when the data are nested in a hierarchical structure, particularly when there are significant differences between higher level units in the dependent variable (e.g., student achievement orientation varies between students, and also between the classes with which the students are nested). In these circumstances, observations are clustered rather than independent. Failure to take the clustering into account leads to underestimation of the errors of parameter estimates, biased significance testing, and a tendency to reject the null when it should be retained. The rationale for using multi level models, as well as thorough explanations of how to carry out the analyses, is provided by

Raudenbush, S. W. Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods. 2nd edition. Newbury Park , CA : Sage.

The R & B book is also well integrated with the authors' HLM software package, which helps a great deal in learning the package. An explanation of why multi-level models are necessary and preferable to some alternatives (like dummy coding the higher level units) is provided in a classic paper

Hoffman, D.A. (1997). An overview of the logic and rationale of Hierachical Linear Models. Journal of Management, 23, 723-744.

The Hoffman paper can be downloaded for free if you Google "Hoffman 1997 HLM" and access the pdf online.