What important ideas came since Nelder and McCullagh's book Generalized Linear Models (a 40 year old book)?

Question

I read not too long ago Nelder and McCullagh's book Generalized Linear Models and thought the book was fantastic and I consider it a useful manual on the subject. Not surprising that's the case, considering Nelder's one of the authors.

However, the book is 40 years old, and surely things have changed since when the book came out. Simon Wood's book, Generalized Additive Models, is very good, but the book focuses on GAMs, not GLMs. (Yes, GAMs generalize GLMs, but I do think focusing on GLMs specifically is worthwhile.)

Hence, what would be the most important developments since Nelder and McCullagh's book came out regarding GLM theory and application? What am I missing from just reading that book? How should I supplement my knowledge?

Answer 1

Your premise that the elapsing of 40 years means that "surely things have changed" is quite dubious in a field relating to applied mathematics. In mathematical work it is often the case that early research on a model form provides all of its essential properties and theory pretty well, and then subsequent research makes smaller innovations/additions with diminishing marginal returns. We are still using mathematical rules from Euclid's Elements (circa 300BC) in many applied mathematical problems today, and while new geometries have been developed since this time, the subject certainly hasn't been transformed substantially every 40 years hence. In any case, I'll try to give you a rough guide to what I think are the most important developments in the field since Nelder and McCullagh's book.

In my view, the biggest change that has occurred in the field since this time is not so much extension of theory for GLMs (though there have been some marginal advances), but the further development and popularising of competing models, some of which are extensions of GLMs and some of which are contrary model forms. In particular, the past 40 years has seen a rapid increase in the use of GLMMs and copula methods.

• Generalised Linear Mixed Models (GLMMs): Generalised linear mixed models (GLMMs) provide an extension of GLMs where there are added "random effects". You can find a nice review of these models in Dean and Nielsen (2007). These models were popularised in the statistics profession in the 1990s with a series of publications showing fitting and inference methods (see e.g., Breslow and Clayton 1993, Breslow and Lin 1995, Lin and Breslow 1996 and Lin and Zhang 1999). Later work in the 2000s gave good overviews of these models, including several textbooks on the subject. These models are now seen as a useful extension of GLMs that can allow for simpler modelling of correlated errors based on explanatory variables in the model. They are now widely used in applied statistics work in a range of fields and are usually included in university programs in statistics.

• Copula methods: Copula methods were first introduced in Sklar (1959) but they didn't really start being used until later. The first statsitical conference on copula methods occurred in 1990 and they started being used more in finance after they were popularised by Li (2000). It is only within the last few decades that copula models have become broadly known in the statistics profession, and probably only in the last decade or so that they've begun to creep into university programs. These models are now presented as an alternative means of modelling the kinds of problems that might previously have been modelled using GLMs.

Some other major changes are set out in the other (excellent) answers below.

Answer 2

In addition to Ben's great answer (+1):

Penalised regression models ($L_1$, $L_2$, elastic net, SCAD (Smoothly clipped absolute deviation), LARS (least-angle regression), MCP (Multiple Change Points), PCR (Principal Component Regression), PLS (Partial least squares) etc.) really came to the fore and became especially relevant for $n\ll p$ applications. While it can be argued that Hoerl & Kennard's Ridge regression: Biased estimation for nonorthogonal problems came out in 1970 (and actually Wold's work on PLS was in 1966), it was until the last 20 to 25 years that most of the adoption and implications of these techniques became prominent within Statistics programmes and curricula.

Answer 3

To add to Ben's great answer, since Nelder and McCullagh's GLMs (1983) is mentioned, I think it's fair to say that one important development after that was the extension of the GLM idea to other families of distributions, that generalize the single parameter exponential family.

Important examples include both Beta and Dirichlet regressions, which are not GLMs in the N&M sense (see Why Beta/Dirichlet Regression are not considered Generalized Linear Models?). Both are important to analyze continuous compositional data, in univariate and multivariate senses, respectively, as alternatives to log-ratio analysis. Here are the respective references:

• Ferrari, Silvia, and Francisco Cribari-Neto. "Beta regression for modelling rates and proportions." Journal of applied statistics 31, no. 7 (2004): 799-815.
• Hijazi, Rafiq Hamed. Analysis of compositional data using Dirichlet covariate models (thesis). American University, 2003.

The Bayesian treatment of conditional models is much more direct, but only more recently the machinery (i.e. software and hardware) made it accessible to be used by most (BUGS was release in the 90s, for example). I cannot pinpoint a single reference for this however, but reading on Bayesian GLMs can be of help.

Answer 4

Hence, what would be the most important developments since Nelder and McCullagh's book came out regarding GLM theory and application? What am I missing from just reading that book? How should I supplement my knowledge?

Numerical methods if you want to implement the theory on your own. Computers are now more powerful and computational methods have advanced.

For specific examples, Firebug mentioned this in their answer about Bayesian methods. Bolker et al. (2009) include a high-level overview of different methods in their article about GLMMs. Personally, I would look at two specific areas for numerical methods:

1. The approach used for speed and larger datasets. For example, Stan is much faster than JAGS for Bayesian methods under most situations. Likewise, Julia can be quicker than R. Both Stan and Julia were created as response to people wanted quicker languages.
2. The assumptions of different GLM methods for ideas such as degrees for freedom and treatment of variance structures.

My own experience with Stan has shown me that mathematical ideas can be sped up in computer languages using methods that seem strange to me and not counter intuitively. Browse the Stan forum for discussion and examples such as the "folk theorem".