Optimise Random Forest Model using GridSearchCV in Python

Question

I am working on a classification problem where I am applying various machine learning models.

I have used DecisionTreeClassifier from Sklearn on my dataset using the following steps:

• Calculated alpha values for the decision tree using the cost_complexity_pruning_path method
• Used GridSearchCV to identify best ccp_alpha value and other parameters. I specified the alpha value by using the output from the step above.

When I review the documentation for RandomForestClassifer, I see there is an input parameter for ccp_alpha. However I am confused on how the alpha value for pruning can be determined in Random Forest.

My understanding of Random Forest is that the algorithm will create n number of decision trees (without pruning) and reuse the same data points when bootstrap is True (which is the default value). The model will predict the classification class based on the most common class value from all decision trees (mode value).

The decision trees in random forest will not be same (generally speaking as that is how the algorithm is designed) and therefore the alpha values for the corresponding decision trees will also differ.

I have 2 questions:

1. How can you determine the ccp_alphas value in RandomForestClassifier?
2. Can you conduct hyperparameter tuning for ccp_alpha value using GridSearchCV for RandomForestClassifier? If yes how can you set this up?

Answer 1

The answer to your questions are both Yes. For 1. Consider that you have a trained classifier, then you just need to do what is explained in this link tutorial. For what concerns the second question, if you have in mind values of this parameter and store them in a dictionary, where the key is named “ccp_alpha”, you will be able to grid search the values. This is feasible since ccp_alpha is a parameter of RandomForestClassifier, see scikitlearn page for classifier.. You would then need to feed GridsearchCV with your classifier.