Difference between OOB score and score of random forest model in scikit-learn package?

Question

I'm using the scikit-learn package to run random forest (RandomForestRegressor) in python. A section of my code looks like this:

clf = RandomForestRegressor(n_estimators=100, oob_score=True)
clf.fit(y, x)
print clf.score(y, x)
print clf.oob_score

Execution of the code gave me two different numbers corresponding to the third and fourth lines (0.945098 and 0.596445, respectively). My understanding is that the first printed measure is called "score" and it is equivalent to R2 in linear regression and the second printed mearure is "OOB score" and it is related to the error rate. Am i correct? If so, can these two measures be used interchangably when evaluating the performance of random forest models? IE, can i simply stick with one of them when comparing models

Thanks!

Answer 1

clf.score(y, x) provides the coefficient of determination (R**2) for the trained model on the given data. Since you pass the same data used for training, this is your overall training loss score. If you would put "unseen" test-data here, you get validation loss.

clf.oob_score provides the coefficient of determination using oob method, i.e. on 'unseen' out-of-bag data. This score serves as cross-validation loss and accordingly to L. Breinman oob_score = cross-validation score https://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm.