# How is it called when a MLR algorithm predicts a value beyond the range of the training data set and is there a way to avoid this for Neural Networks?

I use two Machine Learning Algorithms to learn how my target variable [0, 8] is affected by four features, each within a scale of [1, 10]. I am using scikit-learn to do this task for me.

from sklearn.ensemble import RandomForestRegressor

X_train = np.array([[9.0, 8.0, 2.0, 9.0], [7.0, 9.0, 3.0, 8.0], [1.0, 2.0, 8.0, 3.0]])
y_train = np.array([8.0, 8.1, 2.2])

X_test = np.array([[10.0, 9.0, 1.8, 8.5]])

rforest = RandomForestRegressor().fit(X_train, y_train)
print("Random Forest predicts {:3.2f}".format(rforest.predict(X_test)[0]))


As you can see, I chose my X_train values such that high values for index 0, 1 and 3 and low value for index 2 result in a high value for y_train.

Now comes a new sample which the algorithm has not yet seen (X_test). Note how the first feature is 10.0 and thus higher than any other value the algorithm had learned - though still in the valid range. Despite that fact, the result is:

Random Forest predicts 8.07

If I do the same with the standard settings of an MLPR, I get the following:

from sklearn.neural_network import MLPRegressor

# [...]

mlpr = MLPRegressor().fit(X_train, y_train)
print("Neural Network (MLPR) predicts {:3.2f}".format(mlpr.predict(X_test)[0]))


Neural Network (MLPR) predicts 8.61

The prediction of the NN is outside of the range it learned (y_train). It learned that an exaggeration of any of the features results in an exaggeration of the predicted variable.

This may be of interest for many studies, but in my case I am rather restricted in the range of the predictions. A prominent example would be how a NN should predict concentrations of a certain chemical substance, but returns negative concentrations based on the features it learned.

My problem is that I cannot specify the valid range, only that I would like predictions to be within the range it originally learned.

How is such an algorithm called and is there a way to restrict NN or SVRs to the range in which they were trained (just like the random forest does)?