How to make the randomforest trees vote decimals but not binary My question is about binary classification, say separating good customers from bad customers, but not regression or non-binary classification. In this context, a random forest is an ensemble of classification trees. For each observation, every tree votes a "yes" or "no", and the average vote of all trees is the final forest probability.
My question is about modifying the behavior of the underlying trees: How can we modify the randomForest function (of the randomForest package of R) so that each tree votes a decimal instead of a binary yes/no. To better understand what I mean by decimal, let's think about how decision trees work.
A fully grown decision tree has 1 good or 1 bad instance in its terminal nodes. Assume that I limit the terminal node size as 100. Then terminal nodes are going to look like:
Node1 = 80 bad, 20 good
Node2 = 51 bad, 49 good
Node3 = 10 bad, 90 good
Notice, even though Node1 and Node2 vote "bad", their "strength of bad-ness" is severely different. That is what I am after. Instead of having them produce 1 or 0 (which is the default behavior) can one modify the R package so they vote 80/100, 51/100, 10/100 etc?
 A: It is perfectly possible to grow a "probability forest". The methodology in Malley et al. (2012) "Probability machines: consistent probability estimation using nonparametric learning machines." that outlines how this is done and how it compares to standard random forest implementation. In addition, the excellent R package ranger implements this functionality already; just set probability = TRUE when making the function call to ranger. 
A: Simply use predict.randomForest(..., type="prob"). You are doing a Good Thing.
A: This is a subtle point that varies from software to software. There are two main methods that I'm aware of:


*

*Binary leafs - Each leaf votes as the majority. This is how randomForest works in R, even when using predict(..., type="prob")

*Proportion leafs - Each leaf returns the proportion of the training samples belonging to each class. 
This is how sklearn.ensemble.RandomForestClassifier.predict_proba works. In another answer, @usεr11852 points out that R's ranger package also provides this functionality. Happily, I can attest that from my limited usage, ranger is also much, much faster than randomForest.


I don't think that there's an easy way to get randomForest to use the proportional leaf method, since the R software is actually just a hook into a C & FORTRAN program. Unless you enjoy modifying someone else's code, you'll either have to write your own, or find another software implementation.
