0
$\begingroup$

May I know what hyperparameter to tune for random forest classifier to control misclassification? I'm doing a 5-class classification problem and it turns out that most classes are been misclassified for the major class.enter image description here

Currently, I set the following hyperparameters

model = RandomForestClassifier(n_estimators=200,max_depth=7,
              max_features=.4, min_samples_leaf=2)

In total, I have 40 features, so 40% of the features (16-features ) are used to build the trees.

$\endgroup$
1
$\begingroup$

There's a number of things you can try but none are guaranteed to work. Here's a short list of ideas:

  1. There's an entire field called hyper-parameter optimization that you could explore for tweaking the parameters of your forest. Probably the easiest (and reasonably effective) method is a random search. Give some ranges to each parameter and randomly vary the parameters within your given ranges (try maybe 100-1000 times and see what happens). This of course assumes you have have partitioned your data into 3 sets (a training set, validation set--for hyper-parameter optimization, and a final testing set). Generally, you can count on hyper-parameter optimization to improve your results at least a little bit.

  2. Artificially balance (or improve the balance of) your dataset. This can be done in (at least) two ways: under-sampling or over-sampling. To undersample, randomly down select from your larger classes (e.g. since foot and car are a lot larger than the rest, maybe you only train using half the data from each of those classes--chosen at random). You could take this the extreme and say if your smallest class has $N$ elements, then randomly select $N$ elements from each class for training. To oversample, you could duplicate your smaller classes some number of times. If there's a meaningful way to add noise to this oversampling that will probably help (so you don't simply have exact duplicates).

  3. Related to oversamplings, you can use techniques like SMOTE to generate artificial examples of the smaller classes. You have to be a little careful with this because you're obviously inventing training data, but it can be worth a try.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.