0
$\begingroup$

when I use the GradientBoostingClassifier from scikit-learn, I find that there is a parameter max_depth to set, which controls the maximum depth of the regression tree. May I know what exactly does that parameter do? If max_depth=3, does it mean that the construction of a regression tree will stop growing once the tree exceeds a depth of 3? In this case, this parameter is basically used to control the complexity of the regression tree?

$\endgroup$
2
$\begingroup$

You are right. max_depth bounds the maximum depth of regression tree for Random Forest constructed using Gradient Boosting. However, default value for this option is rather good.

To see how decision trees constructed using gradient boosting looks like you can use something like this

from sklearn import tree
from sklearn.externals.six import StringIO
import pydot
import numpy as np

# generate training sample
training_points = np.random.rand(20, 3)
training_values = np.sum(training_points, axis=1) > 0.8 * np.random.rand(20,)

# get decision tree
decision_tree = tree.DecisionTreeClassifier(max_depth=3)
model = decision_tree.fit(training_points, training_values)

# save tree as pdf
dot_data = StringIO() 
tree.export_graphviz(model, out_file=dot_data) 
graph = pydot.graph_from_dot_data(dot_data.getvalue()) 
graph.write_pdf("decision_tree.pdf") `
$\endgroup$
2
  • $\begingroup$ Thanks for the code! Learned some tricks as well :) I'm still learning the concept of Gradient Boosting. May you elaborate why you mention Random Forest is constructed? It that because you have many regression trees (having the same regions but with different weights) in the final decision function? $\endgroup$
    – mintaka
    Mar 20 '15 at 16:56
  • $\begingroup$ I think that we construct a random forest with Gradient boosting - while it is the terminology question, and in many cases authors consider Random Forest and Gradient Boosting as different approaches. $\endgroup$ Mar 22 '15 at 17:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.