# The tanh activation function in backpropagation

In the backpropagation algorithm when the output activation function is tanh and the number of classes is 2 (binary problem), the value obtained at the output layer is in the range between -1 to 1. The cross-entropy error function has log that is applied on the predicted values. Therefore, if one of the output values is a negative number, an invalid operation, namely, log (non-positive number), occurs, rendering the cross-entropy function invalid.

This boils down to the following questions:

• Is it disallowed to set the output activation as tanh?

• Should the output activation always be the softmax even for a binary class
problem?

On a side note, the tanh and the logistic sigmoid are related linearly. Tanh is just the logistic scaled and translated from the $[0, 1]$ to the $[-1, 1]$ interval.
• You are right, however, like I said, the cross-entropy cost function, in python syntax, is cost = -np.sum(Y * np.log(a_output)) (consider the log) where Y is the real output and a_output constitutes the predicted values. Since tanh can yield negative values, then np.log(a_output) can produce a math domain error. This is what is confusing me, the fact that there is a possibility that log is applied on negative predicted values, causing a math error. – Curious Jul 30 '13 at 7:24