I'm having a really hard time picturing a neural network with softmax activation function on its last layer.
For example, say if the last hidden layer has 100 nodes and the output layer had 10 nodes, then each of the output nodes would receive a sum of 100 inputs multiplied by their weights. The softmax activation function would then be applied to sumand and that would form the output.
However, the output is meant to represent the probability of a classification. But how can the probability ever sum up to 1 since the softmax is applied only to the inputs of each node.
Or is it applied to the whole output from the neural network. But if this is the case, then what activation function is applied to the input of each node.