5
$\begingroup$

I want to know is there any way in R/Python to specify to the model to emphasize its learning more on specific subset of data , while it considers the whole data.

For example - i have sales behavior data from 2011 to 2016 and i am predicting likelihood to buy in 2017 - i want the model to emphasize more on 2015-2016 data ( i.e. capture new learning - which may not be very evident when you consider the whole data from 2011 ). I can always build a separate model for for that time period or consider a time year variable for it to capture the effect , but is there some way to specify to the model that focus more on rows ( x to y ) as in give more weight-age to the learning from this subset from whole data.

Improve this question
$\endgroup$
1
  • 2
    $\begingroup$ many many learners allow weights to be associated with the samples. h2o.ai is one of them. The answer is "yes. easily. most of them". From the basic lm to the advanced keras/pytorch there are weights. CART, Ensembles, and SVM. Oversampling works like weights, but costs more compute and memory. It is a bit of an inelegant hack when there is an input called "weights" that was built to do the job correctly. $\endgroup$
    – EngrStudent
    May 3, 2021 at 12:00

2 Answers 2

Reset to default
1
$\begingroup$

One general approach is to try oversampling more/undersampling less important data.

With respect to weights it will depend on the algorihtm.

In Python it seems many algorithms in scikit-learn have it, for example SVMs, Stochastic Gradient Descent classifiers, and Random Forests have it, though unfortunately I can't find general documentation on this parameter.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Since it is time bound data, I have all the data for 2016 and previous years in my train. Hence over sampling won't really help. What I want to do is provide learners be it weak or strong from 2016 to be given more room and lower acceptance threshold, since they are new trends. I'll search for the arguments in scikit learn functions in python, but it'll be great if you can provide an example and which argument in function it is? $\endgroup$
    – Pb89
    Aug 30, 2017 at 0:23
  • 1
    $\begingroup$ Dod you try over/undersampling? Why do you think it won't work? $\endgroup$
    – Jakub Bartczuk
    Aug 30, 2017 at 7:00
  • 1
    $\begingroup$ Did you see examples? They all use sample_weight argument $\endgroup$
    – Jakub Bartczuk
    Aug 30, 2017 at 7:02
  • $\begingroup$ Yes , i saw but how to i emphasize which sample to overweight - or yes , i can manually create those samples . But i can't directly assign a weight specifying say rows x to y ( so that whatever sample a say random forest takes while building trees from this subset of x to y , it assigns more weight ) $\endgroup$
    – Pb89
    Aug 31, 2017 at 1:14
  • 1
    $\begingroup$ @JakubBartczuk as I see it has only class_weight parameter, which does not allow you to specify this and this samples should be weighted more. $\endgroup$
    – Alina
    Apr 16, 2018 at 10:49
0
$\begingroup$

Keras offers the ability to apply weight to every sample invdividually.
You can view the entire keras model api here. The description of the sample_weight parameter of keras "fit" function is as follows:

sample_weight: Optional Numpy array of weights for the training samples, used for weighting the loss function (during training only). You can either pass a flat (1D) Numpy array with the same length as the input samples (1:1 mapping between weights and samples), or in the case of temporal data, you can pass a 2D array with shape (samples, sequence_length), to apply a different weight to every timestep of every sample. In this case you should make sure to specify sample_weight_mode="temporal" in compile().

As far as i know keras is used for neural networks, so this feature is available for those but not for other models like svm or random forest

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.