Derivative of softmax and squared error

Question

I'm trying to understand the derivatives w.r.t. the softmax arguments when used in conjunction with a squared loss (for example as the last layer of a neural network).

---

I am using the following notation:

Softmax argument for class $i$:

$a_i=b_i + W_i \mathbf{h}$

Vector of softmax arguments:

$\mathbf{a} = (a_1,...,a_K)^T$

Softmax probability for class $i$:

$p_i(\mathbf{a}) = \frac{e^{a_i}}{\sum_k{e^{a_k}}}$

$\mathbf{p(a)} = (p_1(\mathbf{a}),...,p_K(\mathbf{a}))^T$

Vector with true labels:

$\mathbf{y} = (0, ... 0,1, 0,...0)^T$

Squared loss:

$L(\mathbf{p(a)},\mathbf{y}) = \sum_i(p_i(\mathbf{a}) - y_i)^2$

---

Now, I want to get the derivative of the loss w.r.t. a particular softmax argument $a_j$. This is what I arrived at:

$\frac{d}{da_j} L(\mathbf{p(a)},\mathbf{y})$

$= \sum_i \frac{d}{da_j} [p_i^2(\mathbf{a}) -2y_ip_i(\mathbf{a}) + y_i^2] $

$= \sum_i \frac{d}{dp_i} [p_i^2(\mathbf{a}) -2y_ip_i(\mathbf{a})] \frac{dp_i}{da_j} $

$=2 \sum_i (p_i(\mathbf{a}) - y_i) p_i(\mathbf{a}) (1_{i=j} - p_j(\mathbf{a})) $

---

Looking at the draft of the Bengio et al. Neural Networks book (Chapter 6, http://www.iro.umontreal.ca/~bengioy/DLbook/), they state this equation:

$\frac{d}{da_j} L(\mathbf{p(a)},\mathbf{y}) = 2 (\mathbf{p(a)} - \mathbf{y}) \odot \mathbf{p} \odot (1 - \mathbf{p}) $

Where $\odot$ is element-wise multiplication.

How should I interprete this? Is there a mistake in my derivation?

Answer 1

Yes, your formula is correct. The formula in the draft chapter was for the sigmoid not for the softmax. We will fix it. Thanks for pointing it out.

-- Yoshua Bengio

Answer 2

The formula you quote from the book

$$ \frac{d p_i}{d a_j} = \sum_i p_i (1 - p_i) $$

cannot be correct because it has no dependence on $j$. Also, the relationship

$$\sum_k p_k = 1$$

implies that

$$ \sum_k \frac{d p_k}{d a_j} = 0 $$

and this doesn't hold for the book's proposed formula.

I think your formula is correct. Here's my derivation:

$$\begin{align} \frac{d p_i}{d a_j} &= \frac{e^{a_j}e^{a_i} - \delta^i_j e^{a_i} \sum_k e^{a_k}}{ \left( \sum_k e^{a_k} \right)^2 } \\ &= \frac{e^{a_j}}{\sum_k e^{a_k}} \frac{e^{a_i} - \delta^i_j \sum_k e^{a_k}}{\sum_k e^{a_k}} \\ &= p_j (p_i - \delta^i_j) \end{align}$$

My $i$ and $j$ are reversed from yours, but it's easy to see that its the same.