I have a very imbalanced big dataset (500000 instances
, 60 features
) which is prone to changes (increase in size and number of features). But what will stay fixed is the imbalance in the classes, this is, class 0
will always be the dominant one. On average, 90%
of the data will be in class 0
, and the rest 10%
in class 1
.
I am interested in classifying as accurately as possible instances with class label 1, so I want to increase its cost of misclassification.
The classifier I chose is RandomForest
and in order to account for the class imbalance I am trying to adjust the weights, then evaluate using StratifiedKFold
and then plotting the corresponding roc_curve
for respective the k fold.
This is the code for my classifier:
clf1 = RandomForestClassifier(n_estimators=25, min_samples_leaf=10, min_samples_split=10,
class_weight = "balanced", random_state=1, oob_score=True)
sample_weights = array([9 if i == 1 else 1 for i in y])
I looked through the documentation and there are some things I don't understand. I tested all these methods but the difference in the evaluation metrics was minimal so I have a hard time identifying which settings optimize my classifier.
Needless to say even though I use weighting the prediction power of my model is very low, with sensitiviy being on average 0.2
These are my questions:
- should
sample_weight
andclass_weight
be used together simultaneously? - between
class_weights = "balanced"
andclass_weights = balanced_subsamples
which is supposed to give a better performance of the classifier - is
sample_weight
supposed to be adjusted always according to ratio of imbalance in the samples? class_weights = balanced_subsamples
andsample_weight
give an execution error when used simultaneously. why?
Also if there is a better approach for evaluating the classifier please do let me know.
class_weight[class_of_datapoint_i]
timessample_weight[i]
. $\endgroup$sample_weights = array([9 if i == 1 else 1 for i in y])
, 2.class_weight={0: 1, 1: 9}
$\endgroup$class_weight
. It's not wrong, it's just unnecessary complicated $\endgroup$