I want to understand the logic behind keeping ReLU as $max(0,x)$ and not $min(0,x)$?
Why do we prefer positive inputs over the negative ones?
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.Sign up to join this community
The weights learned in a neural network can be both positive and negative. So in effect, either form would work. Negating the input and output weights with the $\min$ form gives the same function as with the $\max$ form. The max form is used purely by convention.