I am working on a classification problem where I am applying various machine learning models.
I have used DecisionTreeClassifier from Sklearn on my dataset using the following steps:
- Calculated alpha values for the decision tree using the cost_complexity_pruning_path method
- Used GridSearchCV to identify best ccp_alpha value and other parameters. I specified the alpha value by using the output from the step above.
When I review the documentation for RandomForestClassifer, I see there is an input parameter for ccp_alpha. However I am confused on how the alpha value for pruning can be determined in Random Forest.
My understanding of Random Forest is that the algorithm will create n number of decision trees (without pruning) and reuse the same data points when bootstrap is True (which is the default value). The model will predict the classification class based on the most common class value from all decision trees (mode value).
The decision trees in random forest will not be same (generally speaking as that is how the algorithm is designed) and therefore the alpha values for the corresponding decision trees will also differ.
I have 2 questions:
- How can you determine the ccp_alphas value in RandomForestClassifier?
- Can you conduct hyperparameter tuning for ccp_alpha value using GridSearchCV for RandomForestClassifier? If yes how can you set this up?