Wikipedia defines a Hierarchical GLM as:

Hierarchical linear models (or multilevel regression) organizes the data into a hierarchy of regressions, for example where A is regressed on B, and B is regressed on C.

However, PyMC comes with a "Hierachical GLM" example defined as

sat_t ~ spend + stu_tea_rat + salary + prcnt_take

Why is this model hierarchical? Aren't we regressing sat_t on all the other variables directly? Or am I reading the definition or model specification incorrectly?

Here is the full code and result.

sat_data = pd.read_csv('data/Guber1999data.txt')
with Model() as model_sat:
    grp_mean = Normal('grp_mean', mu=0, sd=10)
    grp_sd = Uniform('grp_sd', 0, 200)

    # Define priors for intercept and regression coefficients.
    priors = {'Intercept': Normal.dist(mu=sat_data.sat_t.mean(), sd=sat_data.sat_t.std()),
          'spend': Normal.dist(mu=grp_mean, sd=grp_sd),
          'stu_tea_rat': Normal.dist(mu=grp_mean, sd=grp_sd),
          'salary': Normal.dist(mu=grp_mean, sd=grp_sd),
          'prcnt_take': Normal.dist(mu=grp_mean, sd=grp_sd)
    glm.glm('sat_t ~ spend + stu_tea_rat + salary + prcnt_take', 
    trace_sat = sample(500, NUTS(), progressbar=False)

scatter_matrix(trace_to_dataframe(trace_sat), figsize=(12,12));

enter image description here


The term "hierarchical" in this example means that it is a hierarchical Bayesian model. It is not a hierarchical GLM in the sense you describe.


Ha! Wikipedia must be reflecting a fight for a cute term. What I am used to thinking as a"hierarchical linear model" is the model with observation units nested in a higher level units (students nested in classrooms, visits nested in patients, etc.), known also as a multilevel model (Wikipedia, this site ) or a mixed model (this site). The models have been around since at least late 1980s, although the term "hierarchical linear model" has become more widely used after this book was published, and there is a software packages by that name.

The sequence of regression models is nothing new. Economists have been working with these since 1950s under the name of "simultaneous equation models". The use of the term "hierarchical regression" to describe the system of regression equations has been on the rise only in the 2010s -- apparently with a generation of people who studied little statistics. If I were to see a simultaneous equation model referred to as a hierarchical regression as a reviewer, I would not stop at rejecting the paper, frankly. If you are propagating a wrong terminology, you are not helping the progress of science.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.