Assume a dataset where the input x is a vector with values in the range [0, 1], and the label y is a single value in the range [0, inf>.

I want to use this data to train a regression neural network. The network has a single output node which produces a regression scalar. Currently, this node has activation x=y, so it may produce negative values even if the label space contains none.

My question is: Does it make sense to train the model using possibly negative predicted values so that it detects a larger error between negative predicted values and true labels? Or should the output be mapped to the label space by, for example, a ReLU activation, so the model only outputs values that makes sense and learns from the error between these values and the true label?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.