24
$\begingroup$

I have a question about semantics that I would like fellow statisticians' opinions on.

We know models such as logistic, Poisson, etc. fall under the umbrella of generalized linear models. The model includes nonlinear functions of the parameters, which may in turn be modeled using the linear model framework by using the appropriate link function.

I'm wondering if you consider (teach?) situations such as logistic regression as a:

  1. Nonlinear model, given the form of the parameters
  2. Linear model, since the link transforms us to the linear model framework
  3. Simultaneously (1) and (2): It "starts" as a nonlinear model, but may be worked with in such a way that allows us to think of it as a linear model

Wish I could set-up an actual poll...

$\endgroup$
25
$\begingroup$

This is a great question.

We know models such as logistic, Poisson, etc. fall under the umbrella of generalized linear models.

Well, yes and no. Given the context of the question, we must be quite careful to specify what we're talking about -- and "logistic" and "Poisson" alone are insufficient to describe what is intended.

(i) "Poisson" is a distribution. As a description of a conditional distribution, it's not linear (and hence not a GLM) unless you specify a linear (in parameters) model to describe the conditional mean (i.e. it's not sufficient merely to say "Poisson"). When people specify "Poisson regression", they nearly always intend to a model that is linear in parameters, and is therefore a GLM. But "Poisson" alone could be any number of things*.

(ii) "Logistic" on the other hand refers to the description of a mean (that the mean is logistic in predictors). It's not a GLM unless you combine it with a conditional distribution that's in the exponential family. When people say "logistic regression" on the other hand, they almost always mean a binomial model with logit link - that does have mean that's logistic in predictors, the model is linear in parameters and is in the exponential family, so is a GLM.

The model includes nonlinear functions of the parameters,

Well, again, yes and no.

The linear in "generalized linear model" says the parameters enter the model linearly. Specifically, what's meant is that on the scale of the linear predictor $\eta=g(\mu)$, the model is of the form $\eta=X\beta$.

which may in turn be modeled using the linear model framework by using the appropriate link function.

Correct

I'm wondering if you consider (teach?) situations such as logistic regression as a:

(I am changing the order of your question here)

Linear model, since the link transforms us to the linear model framework

It's conventional to call a GLM "linear", for precisely this reason. Indeed, it's pretty clear that this is the convention, because it's right there in the name.

Nonlinear model, given the form of the parameters

We must be very careful here, because "nonlinear" generally refers to a model that is nonlinear in parameters. Contrast nonlinear regression with generalized linear models.

So if you want to use the term "nonlinear" to describe a GLM, it's important to carefully specify what you mean - generally, that the mean is non linearly related to the predictors.

Indeed, if you do use "nonlinear" to refer to GLMs, you will get into difficulty not just with convention (and so be likely to be misunderstood), but also when trying to talk about generalized nonlinear models. It's a bit hard to explain the distinction if you already characterized GLMs as "nonlinear models"!

* Consider a Poisson nonlinear regression model, one where there is no $g(\mu)$ for which the parameters enter linearly, so we still have:

$$ Y\sim \text{Poisson}(\mu_x)$$

but for example, where $x$ is age, $Y$ at a given $x$ is observed deaths, and $\mu_x$ is a model for population annual mortality at age $x$:

$$\mu_x = \alpha + \exp(\beta x)\,.$$

(Normally we'd have an offset here for the population at age $x$ which would shift the $\alpha$ term, but we can posit a situation where we observe a constant exposure. Note that both Poisson and binomial models are used for modelling mortality.)

Here the first term represents a constant death rate due to (say) accidents (or other effects not much related to age) while the second term has an increasing death rate due to age. Such a model may perhaps sometimes be feasible over short ranges of later-adult-but-not-senescent-ages; it's essentially Makeham's law (there presented as a hazard function, but for which an annualized rate would be a reasonable approximation).

That's a generalized nonlinear model.

$\endgroup$
  • 1
    $\begingroup$ Thanks for your input. This is what I'm trying to get at. Obviously "linear" is in the name of GLMs. I'm trying to classify those models that are inherently nonlinear (they are nonlinear in the parameters), yet "transformably linear," and thus fall under the GLM framework. I guess maybe I just answered my own question - the best way to refer to them is "transformably nonlinear." $\endgroup$ – Meg Oct 14 '14 at 23:40
  • 2
    $\begingroup$ The more common way to refer to a model which can be rendered linear in parameters by a transformation is "linearizable" (by contrast with "instrincically nonlinear"). I think we need to be clear what it is that's linear (vs what's nonlinear) when discussing the model, and perhaps to also be clear how such things are conventionally referred to, since people need to be able to locate information and also to be understood when discussing them. Someone who talks about GLMs as "nonlinear" is likely to be misunderstood, unless they add the correct qualifiers that make their meaning clear. $\endgroup$ – Glen_b -Reinstate Monica Oct 14 '14 at 23:44
  • $\begingroup$ I agree. I just see it classified as nonlinear regression in texts and have also been taught by my professors that it's nonlinear. I personally find it confusing since we deal with it in the GLM framework, yet I can (sort of, at least) empathize with calling it either. I think I'm going with linearizable/transformably linear and a discussion of how we get from point A to point B (i.e., how we start with a nonlinear function and transform it to the linear framework). $\endgroup$ – Meg Oct 14 '14 at 23:50
  • 1
    $\begingroup$ Yes, I totally understand. While I also empathize with their impulse to do so, if I had their ear I'd caution them against the practice of calling them nonlinear models (at least not without always qualifying the term), for the reasons I outlined above. That's a big part of why I think this is such an important question - people sometimes do call them nonlinear, which I think is fine as long as we're clear about what we're calling nonlinear, since it's not the most conventional way to refer to the models -- when we defy convention, we should do it carefully and deliberately. $\endgroup$ – Glen_b -Reinstate Monica Oct 14 '14 at 23:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.