# Derivative of softmax and squared error

I'm trying to understand the derivatives w.r.t. the softmax arguments when used in conjunction with a squared loss (for example as the last layer of a neural network).

I am using the following notation:

Softmax argument for class $i$:

$a_i=b_i + W_i \mathbf{h}$

Vector of softmax arguments:

$\mathbf{a} = (a_1,...,a_K)^T$

Softmax probability for class $i$:

$p_i(\mathbf{a}) = \frac{e^{a_i}}{\sum_k{e^{a_k}}}$

$\mathbf{p(a)} = (p_1(\mathbf{a}),...,p_K(\mathbf{a}))^T$

Vector with true labels:

$\mathbf{y} = (0, ... 0,1, 0,...0)^T$

Squared loss:

$L(\mathbf{p(a)},\mathbf{y}) = \sum_i(p_i(\mathbf{a}) - y_i)^2$

Now, I want to get the derivative of the loss w.r.t. a particular softmax argument $a_j$. This is what I arrived at:

$\frac{d}{da_j} L(\mathbf{p(a)},\mathbf{y})$

$= \sum_i \frac{d}{da_j} [p_i^2(\mathbf{a}) -2y_ip_i(\mathbf{a}) + y_i^2]$

$= \sum_i \frac{d}{dp_i} [p_i^2(\mathbf{a}) -2y_ip_i(\mathbf{a})] \frac{dp_i}{da_j}$

$=2 \sum_i (p_i(\mathbf{a}) - y_i) p_i(\mathbf{a}) (1_{i=j} - p_j(\mathbf{a}))$

Looking at the draft of the Bengio et al. Neural Networks book (Chapter 6, http://www.iro.umontreal.ca/~bengioy/DLbook/), they state this equation:

$\frac{d}{da_j} L(\mathbf{p(a)},\mathbf{y}) = 2 (\mathbf{p(a)} - \mathbf{y}) \odot \mathbf{p} \odot (1 - \mathbf{p})$

Where $\odot$ is element-wise multiplication.

How should I interprete this? Is there a mistake in my derivation?

Yes, your formula is correct. The formula in the draft chapter was for the sigmoid not for the softmax. We will fix it. Thanks for pointing it out.

-- Yoshua Bengio

The formula you quote from the book

$$\frac{d p_i}{d a_j} = \sum_i p_i (1 - p_i)$$

cannot be correct because it has no dependence on $j$. Also, the relationship

$$\sum_k p_k = 1$$

implies that

$$\sum_k \frac{d p_k}{d a_j} = 0$$

and this doesn't hold for the book's proposed formula.

I think your formula is correct. Here's my derivation:

\begin{align} \frac{d p_i}{d a_j} &= \frac{e^{a_j}e^{a_i} - \delta^i_j e^{a_i} \sum_k e^{a_k}}{ \left( \sum_k e^{a_k} \right)^2 } \\ &= \frac{e^{a_j}}{\sum_k e^{a_k}} \frac{e^{a_i} - \delta^i_j \sum_k e^{a_k}}{\sum_k e^{a_k}} \\ &= p_j (p_i - \delta^i_j) \end{align}

My $i$ and $j$ are reversed from yours, but it's easy to see that its the same.

• It'd be nice to have a non-calculational reason that the $i$ and $j$ are symmetric. – Matthew Drury May 21 '15 at 15:21