1
$\begingroup$

I'm very confused right now. I'm studing the ordered logit and ordered probit regression and I just read that logit is not a linear model but it is logistic model but I also read that both ordered logit and ordered probit are family with Generalized Linear Models. But if logit or ordered logit is not a linear model how can it be categorized into Generalized Linear Model? Can any one explain this in an easy way to me how it makes sense? I would really appreciate if you also could give any source/reference.

$\endgroup$

1 Answer 1

1
$\begingroup$

GLMs do not claim to be linear!

The typical way to write a GLM uses some “link function” that I will denote with $g$.

$$ g\left( \mathbb E [ Y\vert X] \right)=X\beta $$

We can invert the (often nonlinear) $g$ and apply $g^{-1}$ to each conditional expected value given by $X\beta$.

$$ \mathbb E\left[ Y\vert X \right] =g^{-1}\left( X\beta \right) $$

This is nonlinear, but it is nonlinear in a particular way where the conditional expected value can be transformed and then written as a linear combination of features (given in the first equation).

While this does not address your exact question, I hope this brings clarification on what is going on in a generalized linear model.

$\endgroup$
3
  • $\begingroup$ Another way to put this is that a generalised linear model is what you get when you take linear model and generalise it (by adding the link function) so it can handle non-linear relationships. It is, in ways, a stupid name. $\endgroup$
    – Eoin
    Oct 26, 2022 at 11:23
  • $\begingroup$ @Eoin I’d be careful in saying that, as it suggests that linear models only handle linear relationships when they are perfectly capable of using polynomials, splines, and other basis functions to bend and twist the regression “line”. A GLM is more of a particular way to introduce nonlinearity in the parameters that still retains a lot of desirable attributes of linear modeling that nonlinear modeling in general need not have. $\endgroup$
    – Dave
    Oct 27, 2022 at 4:34
  • $\begingroup$ Yeah, that's fair. File my comment under "not quite right, but useful for beginners". $\endgroup$
    – Eoin
    Oct 27, 2022 at 9:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.