Let's say that I want to predict a value Y that ranges between 0 and 1000. I have for this a set of features denoted X.

How would I force a machine learning model to be better on a specific range of my target. For example, if I want my model to be very good on values between 0 and 100, but it is okay if the model is not that good on values between 100 and 1000.

I am curious to know if there exists such technique. I would said that oversampling over the range of values that I am interested in would be good.



  • $\begingroup$ This looks fishy, give some context. $\endgroup$ – user2974951 Sep 21 '18 at 10:51
  • 1
    $\begingroup$ yes oversampling would work fine. sample weights is the 'api' feature you can look for as well. ( so anything that has additive cost function should be able to support, just whether it is implemented or not) eg in scikitlearn the linear regression supports sample weights but not logistic regression ( though internally the underlying algorithms suppoert sample weights) $\endgroup$ – seanv507 Sep 21 '18 at 10:56
  • $\begingroup$ @seanv507, Isn't it in logisitic regression where you can specifiy sample weights and not in regression setting (on sklearn) ? $\endgroup$ – BenDes Sep 21 '18 at 11:42
  • $\begingroup$ @user2974951, what do you mean by fishy ? I can give you an example where you need to be better for small values and not for big one (or the inverse) $\endgroup$ – BenDes Sep 21 '18 at 11:43
  • $\begingroup$ It looks like sample bias. $\endgroup$ – user2974951 Sep 21 '18 at 11:49

What you are describing is weighted regression. See Section 4 in https://see.stanford.edu/materials/aimlcs229/cs229-notes1.pdf for a detailed description.

Like you are hinting, the idea is to weigh samples differently. In the pdf, the points that are close to the point being predicted are weighted higher. What you can do is give higher weights to all your samples whose target value lies in the range you desire. You can also do a soft-weighting where the weight you assign to a sample is inversely proportional to the distance of its target value from your desired range.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.