I have read in the literature and various other questions (as below) that normalising the inputs to have a mean of 0 and a standard deviation of 1 helps the gradient descent optimiser by making a steaper error surface.
However, do the inputs need to be normalised by taking into account the activation function in the network hidden layers? For example, when using sigmoid should the inputs be [0,1]? For tanh, should the inputs be [-1,1]? For relu, is it ok to have negative inputs?