I have a general question about asymmetric costs. In machine learning problems, there are times when the cost of a false positive is different from the cost of a false negative. Accordingly, models should be built differently to account for this asymmetry in costs.

How is this done for a random forest?

Some possible ways are:

  1. Changing the information gain calculated when considering different splits in a given branch of a decision tree to account for asymmetry
  2. Adjusting the threshold from 0.5 within each leaf when assigning the predicted label of a positive class in a given decision tree
  3. Adjusting the threshold from 0.5 within the collection of decision trees when "voting" on the predicted label for the random forest
  4. Using ROC curves and choosing a different threshold than what is typically chosen (typically, the threshold closest to the top-left corner of the ROC graph is chosen as the "ideal")

Which of these way(s) are implemented to account for asymmetric costs, in practice?

  • $\begingroup$ Random Forest Classifiers are generally build to node purity, so (2) is not an option. $\endgroup$ – bi_scholar May 22 at 9:11
  • $\begingroup$ Shouldn't this be done though? I agree the code that random forest is built on generally does not do this, but you could conceivably customize the node purity calculation to take into account different weightings for a positive and negative class. $\endgroup$ – mnmn May 22 at 14:16
  • $\begingroup$ How is that possible? Since the leaf nodes consist of only one class, the ratio of class labels is either 0 or undefined, so applying different weights does nothing. $\endgroup$ – bi_scholar May 22 at 14:55
  • $\begingroup$ (2) You do not want specific threshold values between nodes, you really are after tuning how the tree is built not how the tree evaluates. (3) Voting thresholds would be quite bad too, since one most forests will collect probabilities from the trees rather than 0 or 1s. (1) is implemented pretty much everywhere (and works well), since adding weights to samples is how ada boost is performed. Never thought about (4) need to think on it a while. $\endgroup$ – grochmal May 22 at 15:04
  • $\begingroup$ @grochmal Something to add regarding the points you've made for (3) and (1): First, (3) and (4) are equivalent and common practice. Second, I don't see any reason why Ada Boost is relevant as a justification for why (1) is common practice. $\endgroup$ – bi_scholar May 22 at 15:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.