I have a somewhat theoretical question regarding the "learning" in supervised regression problems.
From my understanding, the "learning" in most ML algorithms contains creating a hypothesis, that best fits the training data. This is done by adjusting the weights associated to the independent variables in such way, that the mean residual size is as small as possible.
Is it possible to give the learning algorithm a certain "freedom" to make error's?
So figuretively speaking, telling the algorithm: this approximation is within 10 % of the original values, that's good enough, move on and concentrate on the next?
So the learning algorithm shouldn't focus on minimizing the MSE of the residuals, but rather on trying to achieve a constant goodness of fit (within a certain tolerance) over all predictions.
I haven't found anything on this topic. Any literature or hints on how to do this in python would be helpful. (I am working with KERAS).