How is the class label applied in the softmax function?

Question

Im reading this paper: Uncertainty in Deep Learning and in it (page4), the softmax loss is defined as

\begin{align*} E(X,Y) = -\frac{1}{N} \sum^N_{n=1} \log(\hat{p}_{n,c_n}), \end{align*}

where $c_n \in \{1,...,D\}$ is the class label for input $n$, and

$\hat{p}_{nd} = \exp(\hat{y}_{nd})/(\sum_{d'} \exp(\hat{y}_{nd'})$ is the element-wise softmax-function applied to model prediction vector $\hat{y}$.

Where does the class label $d$ actually show up in the softmax calculation? I.e. what does $\hat{y}_{nd}$ or $\hat{p}_{n,c_n}$ respectively shall mean?

Thanks

Answer 1

We are given $N$ input-output pairs: $\{(\textbf{x}_1,y_1),\dots,(\textbf{x}_N,y_N)\}$. The $\textbf{x}$'s are input vectors, also known as covariates or predictors, and the $y$'s are the labels. The problem here is to learn how to predict a class label given the vector of inputs $\textbf{x}$.

$\hat{y}_{nd}$ is what brings the $\textbf{x}$'s into the computations. It is also known as the linear predictor and it is given by $\hat{y}_{nd} = \beta_{0d} + \beta_{1d} x_{1n} + \dots + \beta_{pd} x_{pn}$, where the $\beta$'s are regression coefficients (to be estimated) and $x_{1n}, \dots, x_{pn}$ are the elements of the input vector $\textbf{x}_n$.

$\hat{p}_{n,c_n}$ is the probability of the class that sampling unit $n$ belongs to.