What is a good algorithm to match the class distribution of two data sets? I have data generated by a Monte Carlo simulation (let's call it MC) and a second data set (let's call it data), each having events in two classes 0 and 1. I am trying to write an algorithm such that I can match the number of events in class 0 and 1 of MC to data i.e I want to correct MC events by moving them from one class to other such that the ratio of events in the two classes for both data and MC is same. The way I proceeded is: 


*

*Train a GradientBoostingClassifier from scikit ensemble for both data and MC individually(say data_clf and mc_clf)
 mc_clf.fit(X_mc, Y_mc)
 data_clf.fit(X_data , Y_data)

where Y_mc and Y_data is the corresponding class "mc_class" and "data_class" having values 0 or 1 depending on which class they belong to.


*Now, if X_mc is my input variable, use predict_proba to predict the probability of classifier of data and MC using MC inputs ONLY i.e 
 y_mc = mc_clf.predict_proba(X_mc)
 y_data = data_clf.predict_proba(X_mc)


*After this, I try to move the events of MC from one class to another by comparing their probability in data and MC.
 for i in range(0, len(mc)):
     if (mc.loc[i]['mc_class'] == 0): 
         wgt = y_data[i][0]/ y_mc[i][0]
         if (wgt<1): mc.loc[i]['mc_class_corrected'] = 1
         else: mc.loc[i]['mc_class_corrected'] = mc.loc[i]['mc_class'] 


     if (mc.loc[i]['mc_class'] == 1): 
         wgt = y_data[i][1]/ y_mc[i][1]
         if (wgt<1) : mc.loc[i]['mc_class_corrected'] = 0
         else: mc.loc[i]['mc_class_corrected'] = mc.loc[i]['mc_class'] 

Now suppose I have more events in class 0 than 1 in MC as compared to data. So I expect events from class 0 to move to class 1. However, I see that almost >95% of my events in class 0 of MC are moving to class 1, while I was expecting only about 30% of events to move (when compared to the number of events in data and MC). 
Is there any mistake in my approach ?
 A: Here is my educated guess what is off ...
Suppose you have these simplified data sets
X_mc Y_mc
0 1
0 1
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0

X_data Y_date
0 1
0 1
0 1
0 1
0 1
0 1
0 1
0 1
0 0
0 0

As you can see, the class probabilities for class=0 for x=0 are 
Based on Y_mc: 0.8
Based on Y_data: 0.2

So: 
When applying your algorithm, all mc-examples with will be assigned to class 1. Class probability means, that on average for comparable input a certain percentage of the input has class 0, the rest class 1. It is not a winner-takes-it-all mechanism.
You can simulate this (Pseudo code !):
if (mc.loc[i]['mc_class'] == 0): 
    if y_mc[i][0] > y_data[i][0] and random(0,1) < (y_mc[i][0]-y_data[i][0]):
        mc.loc[i]['mc_class_corrected'] = 1
    else:
        mc.loc[i]['mc_class_corrected'] = mc.loc[i]['mc_class']

So in my example, y_mc[i][0]-y_data[i][0] is 0.8-0.2=0.6. That means, that on average 60 % of the examples with class 0 will be assigned to class 1. 
I think this is related to rejection sampling, but this is not my area of expertise. 
