3
$\begingroup$

Let's say that I want to predict a value Y that ranges between 0 and 1000. I have for this a set of features denoted X.

How would I force a machine learning model to be better on a specific range of my target. For example, if I want my model to be very good on values between 0 and 100, but it is okay if the model is not that good on values between 100 and 1000.

I am curious to know if there exists such technique. I would said that oversampling over the range of values that I am interested in would be good.

Thanks,

Benoit

Improve this question
$\endgroup$
9
  • $\begingroup$ This looks fishy, give some context. $\endgroup$
    – user2974951
    Commented Sep 21, 2018 at 10:51
  • 1
    $\begingroup$ yes oversampling would work fine. sample weights is the 'api' feature you can look for as well. ( so anything that has additive cost function should be able to support, just whether it is implemented or not) eg in scikitlearn the linear regression supports sample weights but not logistic regression ( though internally the underlying algorithms suppoert sample weights) $\endgroup$
    – seanv507
    Commented Sep 21, 2018 at 10:56
  • $\begingroup$ @seanv507, Isn't it in logisitic regression where you can specifiy sample weights and not in regression setting (on sklearn) ? $\endgroup$
    – BenDes
    Commented Sep 21, 2018 at 11:42
  • $\begingroup$ @user2974951, what do you mean by fishy ? I can give you an example where you need to be better for small values and not for big one (or the inverse) $\endgroup$
    – BenDes
    Commented Sep 21, 2018 at 11:43
  • $\begingroup$ It looks like sample bias. $\endgroup$
    – user2974951
    Commented Sep 21, 2018 at 11:49

1 Answer 1

Reset to default
0
$\begingroup$

What you are describing is weighted regression. See Section 4 in https://see.stanford.edu/materials/aimlcs229/cs229-notes1.pdf for a detailed description.

Like you are hinting, the idea is to weigh samples differently. In the pdf, the points that are close to the point being predicted are weighted higher. What you can do is give higher weights to all your samples whose target value lies in the range you desire. You can also do a soft-weighting where the weight you assign to a sample is inversely proportional to the distance of its target value from your desired range.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.