# How is the class label applied in the softmax function?

Im reading this paper: Uncertainty in Deep Learning and in it (page4), the softmax loss is defined as

\begin{align*} E(X,Y) = -\frac{1}{N} \sum^N_{n=1} \log(\hat{p}_{n,c_n}), \end{align*}

where $$c_n \in \{1,...,D\}$$ is the class label for input $$n$$, and

$$\hat{p}_{nd} = \exp(\hat{y}_{nd})/(\sum_{d'} \exp(\hat{y}_{nd'})$$ is the element-wise softmax-function applied to model prediction vector $$\hat{y}$$.

Where does the class label $$d$$ actually show up in the softmax calculation? I.e. what does $$\hat{y}_{nd}$$ or $$\hat{p}_{n,c_n}$$ respectively shall mean?

Thanks

• The $y_i$ is either 0 or 1. May 24, 2020 at 10:44
• yes Sir, thats correct for (binary) classification May 24, 2020 at 10:52
• There are $D$ classes. $\hat{y}_{nd}$ for $d \in \{1,\dots,D\}$ are the probabilities for the $n$th sampling unit, and $\hat{y}_{n,c_n}$ is the largest probability (of the $D$ probabilities). May 24, 2020 at 13:09
• Thanks @papgeo for your answer. Your explaination of $\hat{y}_{nd}$ makes sense, thank you! And you mean $\hat{p}_{n,c_n}$ to be the largest probability? May 24, 2020 at 15:21
• If you formulate those insights into an answer, Im willing to accept that. May 24, 2020 at 15:22

We are given $$N$$ input-output pairs: $$\{(\textbf{x}_1,y_1),\dots,(\textbf{x}_N,y_N)\}$$. The $$\textbf{x}$$'s are input vectors, also known as covariates or predictors, and the $$y$$'s are the labels. The problem here is to learn how to predict a class label given the vector of inputs $$\textbf{x}$$.
$$\hat{y}_{nd}$$ is what brings the $$\textbf{x}$$'s into the computations. It is also known as the linear predictor and it is given by $$\hat{y}_{nd} = \beta_{0d} + \beta_{1d} x_{1n} + \dots + \beta_{pd} x_{pn}$$, where the $$\beta$$'s are regression coefficients (to be estimated) and $$x_{1n}, \dots, x_{pn}$$ are the elements of the input vector $$\textbf{x}_n$$.
$$\hat{p}_{n,c_n}$$ is the probability of the class that sampling unit $$n$$ belongs to.