0
$\begingroup$

The OOB score in SK learn Random Forest Regressor gives the R2 score.

Now, from what I know, R2 score is only valid for linear data.

How do I validate the score of my model if my Random forest model is for non linear data?

$\endgroup$
2
  • $\begingroup$ The model "wrapper" classes like GridSearchCV accept a scoring Parameter that allows you to change the scoring function. I presume it works when oob_score=True, although it might be useful to confirm as much and post the answer here. $\endgroup$ Aug 5 '16 at 12:02
  • $\begingroup$ Also, for the record this question belongs on Stackoverflow, and not here. For some reason I don't see the option to vote that it be moved, instead of just closed. $\endgroup$ Aug 5 '16 at 12:04
1
$\begingroup$

RandomForestRegressor man page explains that you get oob_prediction_s for the training cases as well.

You can then calculate e.g. MSE as

((y - rfr.oob_prediction_)**2).sum() / y.size
$\endgroup$
2
  • $\begingroup$ However this won't work inside grid search, pipelines, etc. $\endgroup$ Aug 5 '16 at 12:00
  • $\begingroup$ @ssdecontrol: did you mean to say that in case of this training being part of a grid search there are more elegant solutions? I'd expect GridSearchCV to use normal predictions on cases held out according to the cross validation scheme by the wrapper - so that is a rather different question: that way you evaluate via normal predictions. $\endgroup$ Aug 5 '16 at 12:11
-1
$\begingroup$

You should be able to use OOB. In random forest bagging, we build each tree on part of data. Then we check r^2 in the remaining data not used to build data. So u should be able to use OOB.

$\endgroup$
1
  • $\begingroup$ But again my question is R2 parameter is for linear data but my data is non linear? Can't we get MSE ? $\endgroup$ Aug 4 '16 at 18:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.