I have a small dataset of 1000 rows and 10 features, working on binary classification to predict whether the supplier's likelihood of meeting the target (coded as
For these 1000 rows, I have the label generated based on business rules.
Now during exploratory analysis, I found out the below
C have shown better adherence to target. Meaning, they mostly fulfill/meet their targets (ex: If 10 transactions, they meet target 7 times and fail 3 times)
F have shown poor adherence to target. Meaning, they mostly don't fulfill/meet their targets (ex: If 10 transactions, they meet target 2 times and fail 8 times)
So, I would like to create a new feature called
supplier adherence and store values as
bad and use it during ML model building.
So, my questions are as follows
a) Does creation of new feature (supplier adherence) based on target variable qualifies as data leakage? If no, why? because, am deriving this info from the
target label. Of course, I could have also checked with business owners (who would have shared their own view on quality of supplier which could have also helped me create a feature)
b) After creating this feature, I intend to split the dataset into train and test splits. Am I right to do this?
c) If a new supplier (unseen data) comes in, I believe his adherence should be 0 because we will have no historical data about that supplier to calculate his adherence. So, he is considered to be a good supplier (due to lack of data) Am I right?
Can help me with each of the questions please?
Okay, I have this dataset obtained by merging two data sources. One dataset which contains the datetime when this deal was made. Meaning, supplier signed a deal with us saying he will buy 1000 (target) units. And we have another dataset where we have info about how much he finally bought (600 units) until this point of time. Meaning, the remaining 400 units can be bought in upcoming days/months/year. But as of now, he hasn't fulfilled his target yet. So, we would like to follow up with supplier and ask him to book remaining orders early. If we don't follow up, they may or may not book
So, based on above, each supplier can have multiple deals (for different products) with us. They (show signs of) fulfill some and they don't (show signs of) fulfill some. I mention signs because, these deals are still in progress (and not closed). So, we consider a threshold of 50%. If supplier booked 50% of target quantity, we consider it is fulfilled (because in real time, no one sticks to 100%)
These deals have dates like deal date which indicates when was this made...ex: we have deals from 2017, 2018, 2019,2020,2021 etc...We also have dates on what was his latest order booking date (for a matching deal)