Can a random forest be 100% accurate using only 1 predictor that is not linearly separable?

Question

A random forest classifier is reporting perfect classification accuracy when I pass it the data that it was trained on even though it has only 1 predictor that with overlapping values between classes.

Is this possible or am I making a error? If it is possible, how?

The distributions of the values in the two classes are shown below.

I know passing training data doesn't give meaningful results on how good the classifier is, and that using only 1 feature in a random forest is strange, but I am trying to assess whether the classifier is overfitting by sequentially adding more features to the classifier and looking at the accuracy on the training and the test set.

Answer 1

Oh yes, that can certainly happen.

One advantage of random forests is that they can model nonlinearities in your data. Thus, they can in principle classify perfectly even if your data are not linearly separable. It is enough if they are nonlinearly separable.

Here is an example in R:

set.seed(1)
xx <- runif(100,-1,1)
yy <- as.factor(xx^2>0.3)

plot(as.numeric(yy),xx,xlab="",xaxt="n",pch=19)
axis(1,1:2,levels(yy))

library(randomForest)
model <- randomForest(yy~xx)
model$confusion

      FALSE TRUE class.error
FALSE    59    0           0
TRUE      0   41           0

Note how the predictor cannot be linearly separated, but nonlinearly: