sklearn - combining decision trees in a Random Forests

Question

I have a decision tree algorithm running on a microcontroller to do real time classification. I transpiled it from a sklearn decision tree into C . I now want to try a random forest and I need to understand how the classifications from each tree in a forest are combined into a single result. I assume that if a data point has different classifications across the trees then the entropy or gini values are combined/compared in some way? Or is there another mechanism? Thanks for your help.

Answer 1

The most common way of combining results in an ensemble of desicion trees for classification is by majority vote. This is why you very often see odd numbers of trees, so that there can never be a tie.

Some example code, based on https://github.com/jonnor/emtrees

int32_t votes[EMTREES_MAX_CLASSES] = {0};
for (int32_t i=0; i<forest->n_trees; i++) {
    const int32_t _class = emtrees_tree_predict(forest, forest->tree_roots[i], features, features_length);
    if (_class >= 0 && _class < EMTREES_MAX_CLASSES) {
        votes[_class] += 1;
    } else {
        return -EmtreesInvalidClassPredicted;
    }
}

int32_t most_voted_class = -1;
int32_t most_voted_votes = 0;
for (int32_t i=0; i<EMTREES_MAX_CLASSES; i++) {
    if (votes[i] > most_voted_votes) {
        most_voted_class = i;
        most_voted_votes = votes[i];
    }
}

return most_voted_class;

One can also support weighting of the votes, though this is more common in ensemble methods with heterogeneous estimators.