0
$\begingroup$

I am training a classification random forest for object detection in images. I have several features (like HoG, edge features etc) which work good enough separately. But when I train using all features together, the results don't improve. E.g. area under curve are as follows:

HoG Features: 0.90

edge Features: 0.81

Combined together: 0.86

I am using scikit-learn random forest library, # of trees = 200, information gain = 'entropy', 2 classes and I have 4000 training examples.

Improve this question
$\endgroup$
  • $\begingroup$ Area under what curve? 200 trees seems small. How many features? $\endgroup$ – Sycorax♦ Apr 5 '16 at 2:23
  • $\begingroup$ sensitivity vs specificity. Hog features = 2500, Edge Features = 2700. Trees size kept small due to time constraints $\endgroup$ – Azhar Apr 5 '16 at 2:42
  • 2
    $\begingroup$ It's plausible that your forest isn't diverse enough; try adding more trees. Moreover, by setting minimum node size to be larger, you can get calibrated probabilities while reducing training time per tree. $\endgroup$ – Sycorax♦ Apr 5 '16 at 2:44
  • $\begingroup$ Increased the tree size to 100, no improvement $\endgroup$ – Azhar Apr 5 '16 at 20:18
  • 1
    $\begingroup$ If you had 200 trees before, 100 trees is a decrease. $\endgroup$ – Sycorax♦ Apr 5 '16 at 20:19
1
$\begingroup$

Here's one possible explanation, though I don't know for sure it's what's happening here:

Remember that a random forest is composed of trees, which are composed of splits, and that each of those splits see a random subset of the input features. In sklearn, the default is that each tree sees the square root of the total number of features.

So, when you only input HoG features, each tree sees some random subset of the HoG features, and can do its thing pretty well.

When you input both edge features and HoG features, each of your trees' splits is going to get to look at some combination of HoG and edge features. If the edge features are just less useful than HoG, as it seems they might be, then each of these splits is going to have fewer chances at finding that one delicious cut of HoG to work with – so each split is going to just be a little worse.

You could try increasing max_features to combat this.

Improve this answer
$\endgroup$
  • $\begingroup$ This did not work $\endgroup$ – Azhar Apr 5 '16 at 21:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.