1
$\begingroup$

I am working on a classification problem where I am applying various machine learning models.

I have used DecisionTreeClassifier from Sklearn on my dataset using the following steps:

  • Calculated alpha values for the decision tree using the cost_complexity_pruning_path method
  • Used GridSearchCV to identify best ccp_alpha value and other parameters. I specified the alpha value by using the output from the step above.

When I review the documentation for RandomForestClassifer, I see there is an input parameter for ccp_alpha. However I am confused on how the alpha value for pruning can be determined in Random Forest.

My understanding of Random Forest is that the algorithm will create n number of decision trees (without pruning) and reuse the same data points when bootstrap is True (which is the default value). The model will predict the classification class based on the most common class value from all decision trees (mode value).

The decision trees in random forest will not be same (generally speaking as that is how the algorithm is designed) and therefore the alpha values for the corresponding decision trees will also differ.

I have 2 questions:

  1. How can you determine the ccp_alphas value in RandomForestClassifier?
  2. Can you conduct hyperparameter tuning for ccp_alpha value using GridSearchCV for RandomForestClassifier? If yes how can you set this up?
$\endgroup$
0
$\begingroup$

The answer to your questions are both Yes. For 1. Consider that you have a trained classifier, then you just need to do what is explained in this link tutorial. For what concerns the second question, if you have in mind values of this parameter and store them in a dictionary, where the key is named “ccp_alpha”, you will be able to grid search the values. This is feasible since ccp_alpha is a parameter of RandomForestClassifier, see scikitlearn page for classifier.. You would then need to feed GridsearchCV with your classifier.

$\endgroup$
2
  • $\begingroup$ Thanks for the comment. However it does not answer my questions as I understand how to prune a decision tree (link 1 from your answer). In addition I know that you can input ccp_alpha in RandomForestClassifier model (link 2 from your answer). Both your points have been covered/referenced in my question. What I want to understand is how can you prune a RandomForest to determine the ccp_alpha values as a generalised alpha values will not work (as generally speaking each decision tree will be different) and secondly how can this be used with GridSearchCV (for hyper-parameter tuning) $\endgroup$
    – tb08
    Mar 25 '21 at 9:40
  • $\begingroup$ @tb08 I really don’t get what you mean. You can do hyper parameter tuning for grid search like with any parameter. $\endgroup$ Mar 25 '21 at 15:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.