Under what conditions should one use multilevel/hierarchical analysis?

Under which conditions should someone consider using multilevel/hierarchical analysis as opposed to more basic/traditional analyses (e.g., ANOVA, OLS regression, etc.)? Are there any situations in which this could be considered mandatory? Are there situations in which using multilevel/hierarchical analysis is inappropriate? Finally, what are some good resources for beginners to learn multilevel/hierarchical analysis?

-

When the structure of your data is naturally hierarchical or nested, multilevel modeling is a good candidate. More generally, it's one method to model interactions.

A natural example is when your data is from an organized structure such as country, state, districts, where you want to examine effects at those levels. Another example where you can fit such a structure is is longitudinal analysis, where you have repeated measurements from many subjects over time (e.g. some biological response to a drug dose). One level of your model assumes a group mean response for all subjects over time. Another level of your model then allows for perturbations (random effects) from the group mean, to model individual differences.

A popular and good book to start with is Gelman's Data Analysis Using Regression and Multilevel/Hierachical Models.

-
I second this answer and would just like to add another great reference on this topic: Singer's Applied Longitudinal Data Analysis text <gseacademic.harvard.edu/alda/>;. Though it is specific to longitudinal analysis, it gives a nice overview of MLM in general. I also found Snidjers and Bosker's Multilevel Analysis good and readable <stat.gamma.rug.nl/multilevel.htm >. John Fox also provides a nice intro to these models in R here <cran.r-project.org/doc/contrib/Fox-Companion/…;. – Brett Magill Aug 22 '10 at 3:17
Thank you all for your responses :) As a follow up question, couldn't most data be conceptualized as being naturally hierarchical/nested? For example, in most psychological studies the there are a number of dependent variables (questionnaires, stimuli responses, etc...) nested within individuals, which are further nested within two or more groups (randomly or non-randomly assigned). Would you agree that this represents a naturally hierarchical and/or nested data structure? – Patrick Aug 23 '10 at 16:20
If any of you multilevel/hierarchical gurus could spare a few minutes I would be very grateful if you could weigh in on the analysis questions posed in a different post (stats.stackexchange.com/questions/1799/…). Specifically, do you think that the pain perception data outlined in that post would be better analyzed by hierarchical analyses than non-hierarchical analyses? Or would it not make a difference or even be inappropriate? Thanks :D – Patrick Aug 23 '10 at 16:31

The Centre for Multilevel Modelling has some good free online tutorials for multi-level modeling, and they have software tutorials for fitting models in both their MLwiN software and STATA.

Take this as heresy, because I have not read more than a chapter in the book, but Hierarchical linear models: applications and data analysis methods By Stephen W. Raudenbush, Anthony S. Bryk comes highly recommended. I also swore there was a book on multi level modeling using R software in the Springer Use R! series, but I can't seem to find it at the moment (I thought it was written by the same people who wrote the A Beginner’s Guide to R book).

edit: The book on using R for multi-level models is Mixed Effects Models and Extensions in Ecology with R by Zuur, A.F., Ieno, E.N., Walker, N., Saveliev, A.A., Smith, G.M.

good luck

-

Here's another perspective on using multilevel vs. regression models: In an interesting paper by Afshartous and de Leeuw, they show that if the purpose of the modeling is predictive (that is, to predict new observations), the choice of model is different from when the goal is inference (where you try to match the model with the data structure). The paper that I am referring to is

Afshartous, D., de Leeuw, J. (2005). Prediction in multilevel models. J. Educat. Behav. Statist. 30(2):109–139.

I just found another related paper by these authors here: http://moya.bus.miami.edu/~dafshartous/Afshartous_CIS.pdf

-

Here's an example where a multilevel model might be "essential." Suppose you want to rate the "quality" of the education provided by a set of schools using students' test scores. One way to define school quality is in terms of average test performance after taking student characteristics into account. You could conceptualized this as, $$y_{is} = \alpha_s + X_{is}'\beta_s + \epsilon_{is},$$ where $y_{is}$ is the continuous test score for student $i$ in school $s$, $X_{is}$ are student attributes centered at school means, $\beta_s$ is a school-specific coefficient on these attributes, $\alpha_s$ is a "school effect" that measures school quality, and $\epsilon_{is}$ are student level idiosyncrasies in test taking performance. Interest here focuses on estimating the $\alpha_s$'s, which measure the "added value" that the school provides to students once their attributes are accounted-for. You want to take student attributes into account, because you don't want to punish a good school that has to deal with students with certain disadvantages, therefore depressing average test scores despited the high "added value" that the school provides to its students.

With the model in hand, the issue becomes one of estimation. If you have lots of schools and lots of data for each school, the nice properties of OLS (see Angrist and Pischke, Mostly Harmless..., for a current review) suggest that you would want to use that, with suitable adjustments to standard errors to account for dependencies, and using dummy variables and interactions to get at school level effects and school specific intercepts. OLS may be inefficient, but it's so transparent that it might be easier to convince skeptical audiences if you use that. But if your data are sparse in certain ways---particularly if you have few observations for some schools---you may want to impose more "structure" on the problem. You may want to "borrow strength" from the larger-sample schools to improve the noisy estimates that you would get in the small-sample schools if the estimation were done with no structure. Then, you might turn to a random effects model estimated via FGLS, or maybe an approximation to direct likelihood given a certain parametric model, or even Bayes on a parametric model.

In this example, the use of a multilevel model (however we decide to fit it, ultimately) is motivated by the direct interest in the school-level intercepts. Of course, in other situations, these group level parameters may be nothing more than nuisance. Whether or not you need to adjust for them (and, therefore, still work with some kind of multilevel model) depends on whether certain conditional exogeneity assumptions hold. On that, I would recommend consulting the econometric literature on panel data methods; most insights from there carry over to general grouped data contexts.

-
 This is an old thread, but in case you read this: OLS with dummy variables and interactions does not borrow strength as the other techniques you mention, right? I have some data where I've broken my analysis into two parts and used two lm (R linear model) commands to model the two parts. I introduced a dummy variable to indicate the two parts, then used lm again on this "unified" model and the answers are close, but not the same. My question would be: is that answer "better", or simply different because of the algorithm? – Wayne Mar 29 '11 at 15:13 @Wayne: if you used dummies and the full set of interactions in the second, the point estimates should be the same. Standard errors may differ because the second method may presume higher degrees of freedom, but you would want to check whether that is a correct modeling assumption. – Cyrus S Jul 26 '11 at 12:07

Multi-level modelling is appropriate, as the name suggests, when your data have influences occurring at different levels (individual, over time, over domains, etc). Single level modeling assumes everything is occurring at the lowest level. Another thing that a multi-level model does is to introduce correlations among nested units. So level-1 units within the same level-2 unit will be correlated.

In some sense you can think of multi-level modelling as finding the middle ground between the "individualist fallacy" and the "ecological fallacy". Individualist fallacy is when "community effects" are ignored such as the compatibility of a teacher's style with a student's learning style, for example (the effect is assumed to come from the individual alone, so just do regression at level 1). whereas "ecological fallacy" is the opposite, and would be like supposing the best teacher had the students with the best grades (and so that the level-1 is not needed, just do regression entirely at level 2). In most settings, neither is appropriate (the student-teacher is a "classical" example).

Note that in the school example, there was a "natural" clustering or structure in the data. But this is not an essential feature of multi-level/hierachical modeling. However, the natural clustering makes the mathematics and computations easier. The key ingredient is the prior information which says that there are processes happening at different levels. In fact you can devise clustering algorithms by imposing a multi-level structure on your data with uncertainty about which unit is in which higher level. So you have $y_{ij}$ with the subscript $j$ being unknown.

-

Generally, speaking a hierarchical bayesian (HB) analysis will lead to efficient and stable individual level estimates unless your data is such that individual level effects are completely homogeneous (an unrealistic scenario). The efficiency and stable parameter estimates of HB models becomes really important when you have sparse data (e.g., less no of obs than the no of parameters at the individual level) and when you want to estimate individual level estimates.

However, HB models are not always easy to estimate. Therefore, while HB analysis usually trumps non-HB analysis you have to weigh the relative costs vs benefits based on your past experience and your current priorities in terms of time and cost.

Having said that if you are not interested in individual level estimates then you can simply estimate an aggregate level model but even in these contexts estimating aggregating models via HB using individual level estimates may make a lot of sense.

In summary, fitting HB models is the recommended approach as long as you have the time and the patience to fit them. You can then use aggregate models as a benchmark to assess the performance of your HB model.

-
 Thank you for your detailed reply Srikant :) I am not currently familiar with Bayesian analyses, but i is one of the topics that I have been meaning to investigate. Is Hierarchical Bayesian analysis different from the other multilevel/hierarchical analyses discussed on this page? If so do you have a recommended resource for interested parties to learn more? – Patrick Aug 23 '10 at 16:24 From an analytical perspective HB analysis = multi-level models. However, the term multi-level models is used when you have different levels that occur naturally (See the example of @ars). The term HB models is used when you do not necessarily have different levels in the situation. For example, if you are modeling a consumer's response to various marketing variables (e.g., price, adv spend etc) then you may have the following structure at the consumer level: $β_i \sim N(\bar{\beta},\Sigma)$ and $\bar{\beta} \sim N(.,.)$ at the population level. For references: See the other answers. – user28 Aug 23 '10 at 16:35

I learned from Snijders and Bosker, Multilevel Analysis: An introduction to basic and advanced multilevel modeling. It is very well pitched at the beginner I think, it must be because I am a thicko where these things are concerned and it made sense to me.

I second the Gelman and Hill as well, a truly brilliant book.

-

Multi-level models should be employed when the data are nested in a hierarchical structure, particularly when there are significant differences between higher level units in the dependent variable (e.g., student achievement orientation varies between students, and also between the classes with which the students are nested). In these circumstances, observations are clustered rather than independent. Failure to take the clustering into account leads to underestimation of the errors of parameter estimates, biased significance testing, and a tendency to reject the null when it should be retained. The rationale for using multi level models, as well as thorough explanations of how to carry out the analyses, is provided by

Raudenbush, S. W. Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods. 2nd edition. Newbury Park , CA : Sage.

The R & B book is also well integrated with the authors' HLM software package, which helps a great deal in learning the package. An explanation of why multi-level models are necessary and preferable to some alternatives (like dummy coding the higher level units) is provided in a classic paper

Hoffman, D.A. (1997). An overview of the logic and rationale of Hierachical Linear Models. Journal of Management, 23, 723-744.