As the title suggests, I have several features which have values of either -1, 0 or 1. If I feed this data into a neural network where I use
ReLu as the activation function for the hidden layers, would the negative and 0 values pose a problem to the NN?
I have heard about dead neurons where using
ReLu which is a stepwise function, causes any inputs less than or equal to 0 the neuron to stop learning and become dead. So naturally if a NN with activation function
ReLu is fed 0 or negative inputs, those neurons will become dead.
Now my data contains several features with 0 and negative values. What to do in such a case? Should I use
LeakyReLu or some other variation of
ReLu? Or should I transform my data such that only positive values remain?
EDIT 1: If the negative and 0 inputs do not cause dead neurons then what causes dead neurons? Also then why do we have activation functions like
ReLu alone can handle dead neurons?