Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I am building a neural network. Each perceptron in the network uses a sigmoid activation function.
Must I normalize my inputs (which currently range form 0 to 1200)? I ask this because the sigmoid function approaches 1 as the input to it approaches infinity. Hence, if my input is not between 0 and 1, I'm afraid that the sigmoid will always return 1.
The inputs should be scaled to the so-called "active range" of the activation function, or, in other words, the area of the function curve where the derivative of the function is clearly non-zero. This is done for backpropagation to work properly, since it uses activation function derivatives, and ~ 0 derivatives imply extremely small (insignificant) changes to NN weights (no learning). For sigmoid, the active range lies somewhere between -sqrt(3) and sqrt(3). You may scale inputs to that range.
Also, yes: sigmoid will always output values in (0;1), because that's sigmoid's range. You will need to scale NN outputs to the necessary ranges.
