2
$\begingroup$

I have read that it is generally better to use sigmoid function than softmax function at output-layer with cross entropy error function as the output of the node in the output layer should be independent of other nodes present in the output layer.

I am not able to think of an usecase or an example where sigmoid function outperforms softmax function. How does the output being independent help sigmoid function to perform better than softmax?

$\endgroup$
2
$\begingroup$

If each data point has exactly one output label, then the output node in fact should be depending on each other. We want the probability to sum to one over the output nodes.

If the data point has multiple output labels in the sense that there are multiple tasks, we want to predict if a picture has an apple and to predict if a picture has a table. Then there isn't a need for the probability to sum to $1$. The probability of an apple and the probability of a table appearing could be independent of each other.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.