How to "punish" or weight certain (non-linear) regression errors? I want to predict the approx. number of open parking lots for a car park for a given time slot (hour of day / day of week etc.).
Using GradientBoostingRegressor, results seem quite ok so far. However I'm wondering how I could weight/punish certain regression errors.
E.g. I want my model to be able to be pessimistic for predictions where the result should be (close to) zero. It would be ok to sacrifice some accuracy for that.
Consider the following plot of actual vs. predicted values on a test set:

(originally posted on SO on how to do with scikit-learn, 
but the question probably belongs more into this community https://stackoverflow.com/questions/43494677/how-to-punish-or-weight-certain-non-linear-regression-errors)
 A: *

*Use a cost function that penalizes optimistic predictions more than it does pessimistic. 


This might be tricky because you'll have to both figure out the exact function to use and how to feed it to the learning algorithm.


*Use a transformation of target variable. 


I'm guessing you are optimizing the mean squared error of residuals. Consider two examples: 


*

*actual = 1 and predicted = 11

*actual = 100 and predicted = 110


With raw target values both of these cases will contribute equally to the cost function, but it seems like the second case is more forgivable than the first one. However if you use log(y) as your target instead of y then the first case will receive a higher penalty than the second one, namely log(1/11)^2 and log(10/11)^2. This looks somewhat similar to what you want to achieve and is very easy to implement.
Finally, think about the features that the model uses for prediction. The training is probably done on historical data. Is there enough data to distinguish for example summer vs. winter behaviour? How will days of major holidays like Christmas be different? The outliers might also be caused by some one-off events like a hurricane or a celebrity visit =). The worst outliers might be worth looking into with respect to what might have confused the model so much.
