4
$\begingroup$

I'm using the scikit-learn package to run random forest (RandomForestRegressor) in python. A section of my code looks like this:

clf = RandomForestRegressor(n_estimators=100, oob_score=True)
clf.fit(y, x)
print clf.score(y, x)
print clf.oob_score

Execution of the code gave me two different numbers corresponding to the third and fourth lines (0.945098 and 0.596445, respectively). My understanding is that the first printed measure is called "score" and it is equivalent to R2 in linear regression and the second printed mearure is "OOB score" and it is related to the error rate. Am i correct? If so, can these two measures be used interchangably when evaluating the performance of random forest models? IE, can i simply stick with one of them when comparing models

Thanks!

$\endgroup$
1
  • $\begingroup$ You absolutely cannot use the first if you are evaluating the predictive power of your model. This score is computed on the data used to train the model, which is useless for evaluating its predictive power. Also, please use python 3, python 2 is drawing very near to its end of life. $\endgroup$ – Matthew Drury May 24 '18 at 18:10
2
$\begingroup$

clf.score(y, x) provides the coefficient of determination (R**2) for the trained model on the given data. Since you pass the same data used for training, this is your overall training loss score. If you would put "unseen" test-data here, you get validation loss.

clf.oob_score provides the coefficient of determination using oob method, i.e. on 'unseen' out-of-bag data. This score serves as cross-validation loss and accordingly to L. Breinman oob_score = cross-validation score https://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.