Why isn't activation function applied to the output layer of a neural net?

Question

Im reading this paper momentarily, and in it (section 2.1.) the predicted output $\hat{\textbf{y}}$ of a single hidden layer neural network is given by

\begin{align} \hat{\textbf{y}} = \sigma(\textbf{x}\textbf{W}_1)+b)\textbf{W}_2, \end{align}

where $\textbf{x}$ is the input vector, $\textbf{W}_1,\textbf{W}_2$ the corresponding weight matrices and $b$ the bias weights.

Two questions arise for me:

1. Why isn't the activation function applied to the output layer, as in

\begin{align} \hat{\textbf{y}} = \sigma(\sigma(\textbf{x}\textbf{W}_1)+b)\textbf{W}_2) \end{align}

2. Why isn't a bias weight added to the output layer, as in

\begin{align} \hat{\textbf{y}} = \sigma(\textbf{x}\textbf{W}_1)+b_1)\textbf{W}_2 + b_2 \end{align}

Any intuition about this?

Happy weekend, cheers

Answer 1

1. If the target is not constrained, e.g. such as probability values, the final activation is typically dropped.

2. As per with note (2) in the paper, they assume centred target values:

Note that we omit the outer-most bias term as this is equivalent to centring the output

Answer 2

On p. 2 they write that to use this model for regression they use squared loss, so no activation is needed, while for classification it is passed through softmax function and use cross-entropy loss.