Which is a better cost function for a random forest tree: Gini index or entropy?
I am trying to implement random forest in Clojure.
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.Sign up to join this community
As I found in Introduction to Data Mining by Tan et. al:
Studies have shown that the choice of impurity measure has little effect on the performance of decision tree induction algorithms. This is because many impurity measures are quite consistent with each other [...]. Indeed, the strategy used to prune the tree has a greater impact on the final tree than the choice of impurity measure.
Therefore, you can choose to use Gini index like CART or Entropy like C4.5.
I would use Entropy, more specifically the Gain Ratio of C4.5 because you can easily follow the well-written book by Quinlan: C4.5 Programs for Machine Learning.