# Different probability values when using DecisionTreeClassifier and RandomForestClassifier

I'm studying the Random Forests and I made a little example to validate my knowledge. I create two classifiers, one with the DecisionTreeClassifier and another with RandomForestClassifier. After I used a cross-validation with the iris dataset and calculate the score for each one, how you could see in the code below. But when I analyze the values of the print, the results differs, what is a little strange for me. In theory if I have a Random Forest with only 1 estimator, that is not equivalent to a simple Tree ? So i don't need to have the same score?

from sklearn import tree, cross_validation
from sklearn.datasets import load_iris

max_depth = 1
n_estimators = 1

X, y = iris.data[:, :2], iris.target  #take only the first two column values
X, y = shuffle(X, y, random_state=42)

# Standardize
mean, std = X.mean(axis=0), X.std(axis=0)
X = (X - mean) / std

kf_total = cross_validation.KFold(len(X), n_folds=2)

clf1 = tree.DecisionTreeClassifier(max_depth = max_depth)
clf2 = RandomForestClassifier(n_estimators=n_estimators)

scoreForest = cross_validation.cross_val_score(clf1, X, y, cv=kf_total, n_jobs = 1)
scoreTree = cross_validation.cross_val_score(clf2, X, y, cv=kf_total, n_jobs = 1)

print 'forest', scoreForest , 'tree', scoreTree


My print result: forest [ 0.64 0.61333333] tree [ 0.69333333 0.66666667]

## 1 Answer

Random forest is not not just a bunch of trees -- each of them is built on a different resample of objects (i.e. it is a bagging ensemble) and optimisation of splits is clipped to $m$ randomly selected attributes. Both procedures are applied mainly to reduce the correlation of individual trees and thus make voting work, and in context of a single tree they actually hurt its accuracy.

Also, you have set max_depth of a decision tree to 1, which makes it a dumb single-split model rather than a decision tree.

This way you got unrealistically bad accuracies in both cases.

• Interesting. Increasing the max_depth, I noticed that the values are more accurate
– Mike
Apr 13 '15 at 22:45