Assume a dataset where the input x is a vector with values in the range [0, 1], and the label y is a single value in the range [0, inf>.
I want to use this data to train a regression neural network. The network has a single output node which produces a regression scalar. Currently, this node has activation x=y, so it may produce negative values even if the label space contains none.
My question is: Does it make sense to train the model using possibly negative predicted values so that it detects a larger error between negative predicted values and true labels? Or should the output be mapped to the label space by, for example, a ReLU activation, so the model only outputs values that makes sense and learns from the error between these values and the true label?
