0
$\begingroup$

I am currently using Matlab to generate random forests. I am using the TreeBagger class with the function oobError. I can plot a 2D figure which puts my tree count on the X-axis (IKA weak learners count) and classification error is on the Y-axis.

My question is: How can I interpret the actual error of my classifier (something like cross-validation which gives you a double as your classification error)?

Another question is: Is the X-axis of my figure the number of trees in bag or is it describing which tree (tree #50 for example) has the accuracy given at the corresponding point on the Y-axis?

$\endgroup$
1
  • $\begingroup$ what do you mean by "gives you a double"? Are you referring to estimates of the two types of classification errors that can occur in a two-class problem? $\endgroup$ Commented Aug 19, 2012 at 13:34

1 Answer 1

2
$\begingroup$

If you're doing something like the example in the matlab documentation, then the plot contains the Out-Of-Bag Error as a function of the total number of trees.

If you want to cross-validate your model (which you should!), then build the TreeBagger object with some portion of your data and apply it the remaining held-out data with the predict() method. Matlab has a bunch of utility functions to make cross-validation easier. Take a look at the cvpartition class and the crossval function, though it's obviously not too difficult to write your own versions either.

$\endgroup$
2
  • 1
    $\begingroup$ Thanks for your answer, however, I have read many articles and other stuff that the out of bag error is the real measure of the CCR in random forest. The k-fold cross validation method may not be suitable. $\endgroup$
    – Amir Zadeh
    Commented Aug 26, 2012 at 10:13
  • $\begingroup$ Interesting! Looking into this a little bit more, one can do even better by correcting the out-of-bar error rate, as in Bylander (2000) cs.utsa.edu/~bylander/pubs/ml00-final.pdf. $\endgroup$ Commented Aug 26, 2012 at 18:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.