1
$\begingroup$

I have been reading about imbalanced dataset lately and new to data science Domain. I came across this article which talks about various methods to solve this problem in R. Considering e-commerce data, we have Baseline sales(sales without promotion) and Incremental sales(sales with promotion). As this will be a true example of the imbalanced dataset, Supposedly in the hypothetical example, we have 2 million sales record for baselines for a year and 100K sales record for incremental sales. We can't Undersample and oversample as we will loose important information and won't be able to handle a large date with duplicates because of oversampling. If we want to tent the significance of promotion we can use SS3 ANNOVA but we will not have a true reflection. How do we find the lift of promotion or incremental sales over baseline sales in this scenario as CSL(cost sensitive learning) will not balance the data well. Any thoughts on how to handle this scenario? This lift analysis would help to analyze if promotion significantly helped in getting more sales FYI.

And regarding seasonality for year-long data, we can do seasonal adjustment using time decomposition in R and subtracting/dividing based on series.

$\endgroup$
4
  • 1
    $\begingroup$ In my experience, the articles statement that "ml algortihms tremble at imbalanced data" is just false. $\endgroup$ – Matthew Drury Apr 18 '17 at 20:28
  • $\begingroup$ Yes, I can't speak about its authenticity, I have been gathering resources to self-learn new concepts where I can challenge myself. Do you suggest some way go about this problem? $\endgroup$ – Vinz7791 Apr 18 '17 at 20:32
  • $\begingroup$ Not really an answer, but only a hint: you could take a look to the package uplift. Its purpose is not the to assess the significance of the promotion, but rather to build a classifier; however some of its built-in functions might be helpful. $\endgroup$ – lrnzcig Apr 19 '17 at 15:23
  • $\begingroup$ Thanks, I never knew about this modeling technique, it looks complex but I have gathered few resources and will give it a shot and share my results. $\endgroup$ – Vinz7791 Apr 20 '17 at 18:06
1
$\begingroup$

Assuming that the two populations have roughly the same and pretty much uniform characteristics (demographics basically) and you want to test the difference of a numerical variable (revenue gained or ROI or whatever) what I would do if I were you is:

  1. Split the dataset to train and test sets at random 1.4M baseline and 70K exposed to advertising training set and the rest 0.6M and 30K test set

  2. Split the training set into many smaller datasets (with replacement) consisting of a balanced 50-50 sample from both exposed and non exposed to advertising.

  3. Use cross validation to tune the model's parameters (you can try many different kinds of models here) to predicting the difference between your response variable in the two groups.

  4. To assess how accurate your model is split the testing set dataset again into equal advertised and non advertised people but this time without replacement in the non-advertised population and the same people every time in the advertised population (i'm open to different approaches here) and see how well your model predicts the difference between the two populations in these (unseen) data.

  5. Re iterate the above process from step 1, changing the seed every time, and report the average error (difference between predicted and actual value)

$\endgroup$
1
  • 1
    $\begingroup$ Thanks, for the detailed reply. Can you suggest what kind of model I should run? I am thinking about ANOVA/t-test to see variance difference. Also, how will we account for Lift which is equivalent to (Promoted Sales/ Base Sales)*100, won't it be biased based on training data set which has 50-50 sample. $\endgroup$ – Vinz7791 Apr 18 '17 at 20:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.