Definition of softmax function This question follows up on stats.stackexchange.com/q/233658
The logistic regression model for classes {0, 1} is
$$ \mathbb{P} (y = 1 \;|\; x) = \frac{\exp(w^T x)}{1 + \exp(w^T x)} \\
\mathbb{P} (y = 0 \;|\; x) = \frac{1}{1 + \exp(w^T x)} $$
Clearly those probabilities sum to 1. By setting $w = \beta_1 - \beta_0$ we could also define logistic regression as
$$ \mathbb{P} (y = c \;|\; x) = \frac{\exp(\beta_c^T x)}{\exp(\beta_0^T x) + \exp(\beta_1^T x)} \quad \forall \; c \in \{0, 1\} $$
However, the second definition is rarely used because the coefficients $\beta_0$ and $\beta_1$ are not unique. In other words, the model is not identifiable, just like linear regression with two variables that are multiples of each other.
Question
In machine learning, why is the softmax regression model for classes {0, 1, ..., K – 1} usually defined as follows?
$$ \mathbb{P} (y = c \;|\; x) = \frac{\exp(\beta_c^T x)}{\exp(\beta_0^T x) + \dots + \exp(\beta_{K-1}^T x)} \quad \forall \; c \in \{0, \dots, K-1\} $$
Shouldn't it instead be
$$ \begin{align*}
\mathbb{P} (y = c \;|\; x) &= \frac{\exp(w_c^T x)}{1 + \exp(w_1^T x) + \dots + \exp(w_{K-1}^T x)} \quad \forall \; c \in \{1, \dots, K-1\} \\
\mathbb{P} (y = 0 \;|\; x) &= \frac{1}{1 + \exp(w_1^T x) + \dots + \exp(w_{K-1}^T x)}
\end{align*} $$
Side note: In statistics, softmax regression is called multinomial logistic regression and the classes are {1, ..., K}. I find this a bit awkward because when K = 2, the classes are {1, 2} instead of {0, 1} so it's not exactly a generalization of logistic regression.
 A: Yes, you are correct that there is a lack of identifiability unless one of the coefficent vectors is fixed. There are some reasons that don't mention this. I can't speak to why they omit this detail, but here's an explanation of what it is and how to fix it. 
Description
Say you have observations $y_i \in \{0, 1, 2, \ldots, K-1\}$ and predictors $\mathbf{x}_i^\intercal \in \mathbb{R}^p$, where $i$ goes from $1$ to $n$ and denotes the observation number/index. You will need to estimate $K$ $p$-dimensional coefficient vectors $\boldsymbol{\beta}^0, \boldsymbol{\beta}^1, \ldots, \boldsymbol{\beta}^{K-1}$. 
The softmax function is indeed defined as 
$$
\text{softmax}(\mathbf{z})_i = \frac{\exp(z_i)}{\sum_{l=0}^{K-1}\exp(z_l)},
$$
which has nice properties such as differentiability, it sums to $1$, etc.
Multinomial logistic regression uses the softmax function for each observation $i$ on the vector 
$$
\begin{bmatrix}
\mathbf{x}_i^\intercal \boldsymbol{\beta}^0 \\
\mathbf{x}_i^\intercal \boldsymbol{\beta}^1 \\
\vdots \\
\mathbf{x}_i^\intercal \boldsymbol{\beta}^{K-1},
\end{bmatrix}
$$
which means
$$
\begin{bmatrix}
P(y_i = 0) \\
P(y_i = 1) \\
\vdots \\
P(y_i = K-1)
\end{bmatrix}
=
\begin{bmatrix}
\frac{\exp[\mathbf{x}_i^\intercal \boldsymbol{\beta}^0] }{ \sum_{k=0}^{K-1} \exp[\mathbf{x}_i^\intercal \boldsymbol{\beta}^k] } \\
\frac{\exp[\mathbf{x}_i^\intercal \boldsymbol{\beta}^1] }{ \sum_{k=0}^{K-1} \exp[\mathbf{x}_i^\intercal \boldsymbol{\beta}^k] } \\
\vdots \\
\frac{\exp[\mathbf{x}_i^\intercal \boldsymbol{\beta}^{K-1}] }{ \sum_{k=0}^{K-1} \exp[\mathbf{x}_i^\intercal \boldsymbol{\beta}^k] }
\end{bmatrix}.
$$
The problem
However, the likelihood is not identifiable because multiple parameter collections will give the same likelihood. For example, shifting all the coefficient vectors by the same vector $\mathbf{c}$ will produce the same likelihood. This can be seen if you multiply each the numerator and denominator of each element of the vector by a constant $\exp[-\mathbf{x}_i^\intercal \mathbf{c}]$, nothing changes:
$$
\begin{bmatrix}
\frac{\exp[\mathbf{x}_i^\intercal \boldsymbol{\beta}^0] }{ \sum_{k=0}^{K-1} \exp[\mathbf{x}_i^\intercal \boldsymbol{\beta}^k] } \\
\frac{\exp[\mathbf{x}_i^\intercal \boldsymbol{\beta}^1] }{ \sum_{k=0}^{K-1} \exp[\mathbf{x}_i^\intercal \boldsymbol{\beta}^k] } \\
\vdots \\
\frac{\exp[\mathbf{x}_i^\intercal \boldsymbol{\beta}^{K-1}] }{ \sum_{k=0}^{K-1} \exp[\mathbf{x}_i^\intercal \boldsymbol{\beta}^k] }
\end{bmatrix}
=
\begin{bmatrix}
\frac{\exp[\mathbf{x}_i^\intercal (\boldsymbol{\beta}^0-\mathbf{c})] }{ \sum_{k=0}^{K-1} \exp[\mathbf{x}_i^\intercal (\boldsymbol{\beta}^k-\mathbf{c})] } \\
\frac{\exp[\mathbf{x}_i^\intercal (\boldsymbol{\beta}^1-\mathbf{c})] }{ \sum_{k=0}^{K-1} \exp[\mathbf{x}_i^\intercal (\boldsymbol{\beta}^k-\mathbf{c})] } \\
\vdots \\
\frac{\exp[\mathbf{x}_i^\intercal (\boldsymbol{\beta}^{K-1} - \mathbf{c})] }{ \sum_{k=0}^{K-1} \exp[\mathbf{x}_i^\intercal (\boldsymbol{\beta}^k -\mathbf{c}) ] }
\end{bmatrix}.
$$
Fixing it
The way to fix this is to constrain the parameters. Fixing one of them will lead to identifiability, because shifting all of them will no longer be permitted.
There are two common choices:


*

*set $\mathbf{c} = \boldsymbol{\beta}^0$, which means $\boldsymbol{\beta}^0 = \mathbf{0}$ (you mention this one), and

*set $\mathbf{c} = \boldsymbol{\beta}^{K-1}$, which means $\boldsymbol{\beta}^{K-1} = \mathbf{0}$.


Ignoring it
Sometimes the restriction isn't necessary, though. For instance, if you were interested in forming a confidence interval for the quantity $\beta^0_1 - \beta^2_1$, then this is the same as $\beta^0_1 - c - [\beta^2_1-c]$, so inference on relatively quantities doesn't really matter. Also, if your task is prediction instead of parameter inference, your predictions will be unaffected if all coefficient vectors are estimated (without constraining one). 
