Note: there is a nearly identical question on Stack Overflow. However, I seem to be missing something... or maybe it's just that Python isn't my first language ;)
For a neural network library, I've implemented a number of activation and cost functions along with their derivatives. Any combination of activation and cost function can be chosen for the network's output layer.
For backpropagation, the output layer's error gradient with respect to each node's input is simply the product of the activation derivative and cost derivative (forgive the pseudocode):
$gradient_j = activation.derivative(y_j) * cost.derivative(y_j, t_j)$
where $y$ is the network's final (activated) output and $t$ is the target output.
How can I implement my Softmax derivative to work here? So far, all activation gradients have been computable using only the $y$ value: logistic, tanh, etc. Softmax appears to be a special case; can someone help me out?