# Random Forest Classifer - Performance Evaluation

I have a random forest model that attempts to classify a binary outcome (either "Good" or "Bad"). I am currently looking at ways in which I can evaluate the performance of the model when it is used on newer data to determine if the model is still performing acceptably.

The only performance metrics I can think off are:

• Gini
• Comparison of average actual vs predicted "bad rate"

Are there any other tests that are used to monitor how well a random forest model is performing?

Note: In another question ( Random Forest - Variable Importance over time ) I tried asking about how to explain the predictions of a random forest model. However, this question is not about model interpretability. My main goal here is to find a set of performance metrics that can help determine whether my model needs to be re-trained or not.

• What's wrong with your garden-variety classification quality measures like accuracy, F1, AUROC, etc? Aug 27, 2016 at 12:57
• I mentioned accuracy in my original post (actual vs predicted), likewise area under ROC curve is a linear transformation of gini. I have no problem in using the standard measures (forgot to add confusion matrices) - but I was wondering if there are any other non-standard techniques that others have found superior that I might be missing. Aug 27, 2016 at 13:03
• Area under the ROC curve is NOT a linear transformation of Gini. Gini index measures the within-node variance, ROC measures something different. I don't think you can easily compute Gini index on test observations, though. Mean squared error loss on predicted probabilities seems like a good choice then. Feb 25, 2021 at 12:42
• I think the confusion here is that I was referring to the Gini coefficient whereas your comment is in reference to the Gini measure of impurity that is used in building decision tree nodes. en.wikipedia.org/wiki/Decision_tree_learning#Gini_impurity The gini coefficient is commonly used to measure the rank-ordering performance of a model and I believe it is linearly related to the area under the ROC curve. Feb 26, 2021 at 13:24