I’m trying to better understand unsupervised random forests. An important part of understanding unsupervised random forests is being able to assess how good / appropriate a given forest is. For instance Breiman's page on unsupervised forests says:
If the oob misclassification rate in the two-class problem is, say, 40% or more, it implies that the x -variables look too much like independent variables to random forests. The dependencies do not have a large role and not much discrimination is taking place. If the misclassification rate is lower, then the dependencies are playing an important role.
However, the R implementation does not return a misclassification rate in such an unsupervised forest. It does however return a ‘votes’ component which, in the supervised case, can be readily translated into a confusion matrix and a misclassification rate. The thought is that examining this votes component should help understand the misclassification in the unsupervised forest. However, I’m confused as to what these votes actually represent.
My understanding is that, in the unsupervised case, classification is being done between the actual data and a randomised dataset something like this:
set.seed(23) require(dplyr) require(randomForest) n <- nrow(iris) irisBS <- mutate_each(iris,funs(sample(.,replace=TRUE))) y <- factor(c(rep(1, n), rep(2, n))) rfPU <- randomForest(x=rbind(iris,irisBS), y=y)
As expected, this returns a votes component with twice the number of rows of the original data (iris):
n #  150 nrow(rfPU$votes) #  300
However, the actual unsupervised forest returns a votes component with the same number of rows as the original data:
set.seed(23) rfU <- randomForest(x=iris) nrow(rfPU$votes) #  150
The question is then what these represent. The total votes for class two in the actual unsupervised forest are somewhere between those in my pseudo-unsupervised version above when either the correct or incorrect classification is considered:
sum(rfU$votes[,2]) #  87.12367 sum(rfPU$votes[1:n,2]) #  24.34562 sum(rfPU$votes[(n+1):(2*n),2]) #  119.3299
That is, the value from the unsupervised forest is somewhat above the midpoint of the two other values, but it’s not obvious to me what’s going on. Doing all this many times suggests this is representative.