Is logistic regression a generalized linear model for more than two classes?

Question

Suppose that $K>2$ denote the number of classes and $Y$ be the class label that follows the logistic regression model for a given $X \in \mathbb{R}^d$ as follows: $$ P(Y=k|X)= \frac{e^{\beta_k^\top X}}{1+\sum_{i=1}^{K-1} e^{\beta_i^\top X}}, \quad k=1,\ldots,K-1 \\ P(Y=K|X)=\frac{1}{1+\sum_{i=1}^{K-1} e^{\beta_i^\top X}}. $$ It is well known that for $K=2$, the logistic regression model is a GLM (generalized linear model) and the logit is the cannonical link function for the exponential family. My question is does the same conlcusion still hold for multi-class?

In other words, can we find an exponential family distribution, a link function and suitable set of parameters such that the logistic regression model above falls into a GLM category?

My attempt: With a slight abuse of notation, denote $Y \in\mathbb{R}^{K-1}$ as the one-hot encoding of the class labels in $\{1,\ldots,K-1\}$. Then we have $$ P(Y|X)= \exp \{ Y^\top(\log(\frac{p_1}{p_K}),\ldots,\log(\frac{p_{K-1}}{p_K})) +\log p_K \}, $$ where $p_i$ denotes the class probabilities for each $X$. I am not sure if this satisfies the criteria for being a GLM.

Answer 1

Short answer is YES. But you need to extend the concept of an exponential family of distributions to vector-valued distributions. (And for the extension to be useful it should inherit at least some of the important properties of exponential families in the scalar case). Such an extended concept is discussed at https://en.wikipedia.org/wiki/Exponential_family#Vector_parameter.2C_vector_variable and a textbook treating models based on such vector exponential families is Ludwig Fahrmeir, Gerhard Tutz: "Multivariate Statistical Modelling Based on Generalized Linear Models (Second Edition)" (Springer). The multinomial logit model you discuss in the question, is discussed there as an example. But in R, the glm function can only fit models based on the scalar exponential family, and I don't think there is any unified implementation for the vector case in R.

A very complete treatment of exponential families (both scalar and vector case) is in Lawrence D. Brown: "Fundamentals of Statistical Exponential Families with Applications in Statistical Decision Theory".