Is multinomial logistic regression really the same as softmax regression

Question

Multinomial logistic regression (MLR) is an extension of logistic regression for more than $2$ classes. The extension is made up by keeping linear boundaries between classes and using the class $K$ as pivot: $$\log \frac{Pr(G=i)}{Pr(G=K)} = \beta_i x$$

Now since everything has to sum up to 1:

$$\sum_{i=1}^K Pr(G=i) = 1\Rightarrow \sum_{i=1}^{K-1} e^{\beta_i x}Pr(G=K) +Pr(G=K) \Rightarrow Pr(G=K) = \frac{1}{1+\sum_{i=1}^{K-1} e^{\beta_i x}}$$

Softmax on the contrary assumes for all classes:

$$Pr(G=i)= \frac{1}{C}e^{\beta_i x}$$

where $C$ is a constant. Forcing to sum up to one:

$$C= \sum_{i=1}^K e^{\beta_ix}$$ so: $$Pr(G=i)= \frac{1}{\sum_{i=1}^K e^{\beta_ix}}e^{\beta_i x}$$

Now here are the things that aren't clear to me:

• How are they said to be the same if they do not even have the same parameters? By using class $K$ as pivot, MLR does not have parameters $\beta_K$, while Softmax has.
• If they are the same, can someone prove to me?
• If they aren't the same, I assume the boundaries cannot be the same: are they at least similar?

Answer 1

Softmax and logistic multinomial regression are indeed the same.

In your definition of the softmax link function, you can notice that the model is not well identified: if you add a constant vector to all the $\beta_i$, the probabilities will stay the same. To solve this issue, you need to specify a condition, a common one is $\beta_K = 0$ (which gives back logistic link function). But you can of course specify something else, like the sum of $\beta_i$ to be $0$ for example. Then the parameters of the softmax regression will be different from the parameters of the logistic regression, but there will be a one to one transform to go from one to another. Meaning that making inference with one model is equivalent.