I've been training a fully connected neural network I've developed so that it can learn the XOR problem. I got succesful results using hyperbolic tangent and ReLU as activation functions, this is, the network output matched with the outputs of the XOR truth table. Still, as far as I understand, the activation function should be chosen taking into account the input range, which in this case is $[0, 1]$. As that range is the active input range of the logistic function I wanted to use the latter as activation function.
Using the logistic function I get completely random results, as thay are close to $0.5$ in all cases, i.e. any input combination of $0$'s and $1$'s results in a value close to $0.5$. This leads me to think that the each output is just a guess.
What I don't understand is why if my input is bounded in the $[0, 1]$ range it works with activation functions with output range of $(-1, 1)$ or $[0, +inf)$ and not $(0, 1)$? Does my reasoning make sense or am I missing something?
Thanks in advanced.
EDIT: I've tested another set of outputs for the same group of inputs, more specifically
inputs = [[0, 0], [0, 1], [1, 0], [1, 1]] and
outputs = [[1, 1], [1, 0], [0, 1], [0, 0]], and get correct results:
[0, 0] --> [ 0.99999543, 0.99488362]
[0, 1] --> [ 9.67808797e-01, 4.01490200e-04]
[1, 0] --> [ 3.19309525e-05, 9.92688220e-01]
[1, 1] --> [ 0.0216361, 0.0097268]
Other cases with 2 or more outputs work as well, but I still can make the XOR problem, with one output, work. Why would the network, using logistic functions as activations, classify samples correctly when having 2 or more outputs and not when there is only one?