Neural network regression with constraint

Question

I am training a neural network for a regression task, where the dependent variable varies in the range from $0$ to $10$. Unsurprizingly, with the test data set, I obtain the predictions that fall outside of this range. Two approaches come to mind:

1. Rescale the output to interval $[0,1]$ and use a neural network classifier, where the probability of output $1$ is taken as the value of the dependent variable. This requires properly defining the loss function - it easily could be done in a hand-made network, but I am not sure whether standard Python packages are adaptable for this purpose (I have tried only sklearn so far, but I am open to suggestions.)
2. Use a regression network, but modify the loss function to limit the output to the required interval. Here, it is not clear what loss function would work best (mathematically and from the computational viewpoint).

I will appreciate advice from those who have dealt with a similar situation. Specifically: 1. What is the best method for dealing with this problem? 2. How could I adapt an existing tool (anything that works under python)?

Answer 1

Similar question was asked before. In case of constraints like this, there is an easy solution: just transform the outputs to fit the range. In neural networks, we use activation functions on the output layer to achieve this. For example, if you needed map the outputs to $[0, 1]$, you would use a sigmoid activation function. Here, you can just use sigmoid(h) * 10, or something else, as activation function on the final layer. Technically, this is the same as dividing the predicted values by 10 and using standard sigmoid activation, as you proposed.

If you used unconstrained regression and "some kind" of regularization, this would still not guarantee the outputs to fit the constraint, since regression would still be able to make predictions outside the bounds, so this is not the way to go.