I am constructing classification trees for the first time, so I'm quite new to this use of R.

I have observations of behaviours and incoming data that has to be classified as one of these behaviours. Using the rpart package, I was able to construct a tree:

Tree <- rpart(formula = behaviour ~., data=Tree.Data, method="class")

However, the behaviours I observed do not include all possible behaviours. For example I observed eating and sleeping, but not shaking. Now if my incoming data is actually shaking, I would prefer it to be classified as "unknown" rather than eating (as the best guess of the tree). Additionally, if the tree is not that certain of a prediction (less than 70% sure?), I would like it to be in this "unknown" class too.

I found that I can obtain the probabilities of each incoming datapoint being a certain behaviour:

pred <- predict(Tree, data = incoming.data[,c(behaviour)], type = c("prob"))

However, I do not know how to proceed...

Is there any way within the rpart package to do this? I also looked into Random Forest and thought I might be able to do something with the votes, but I got stuck on this too.

To clarify: I do not want to be able to classify unknown behaviours, I just want to put them on one big "unknown" pile.


1 Answer 1


You won't be able to get a classification of "Unknown" unless you trained your tree on Unknown behaviors.

And even if you did include Shaking and labeled it "Unknown" in your training sample, your tree would only be able to recognize Shaking - and classify it as "Unknown" - but not some other kind of not-yet-seen behavior, e.g., Swinging From A Chandelier While Reciting Shakespeare.

Donald Rumsfeld's quote about the difference between known unknowns and unknown unknowns is relevant.

What you can do is to look at the classification probabilities, as you already do. You can post-process the probabilities and label anything that does not have a, say, 70% probability for any one class as "Unknown".

If you get more data, you can even try to spot patterns in such low-probability classifications. Maybe if your tree outputs about 40% probability of Eating and 40% of Sleeping, then this is really Shaking, but if the probabilities are 60% and 15%, then we have a case of Swinging From A Chandelier While Reciting Shakespeare. Then again, if you have enough data to run this kind of analysis, you might just as well use the full labeled data to train your tree in the first place.

And Random Forests are a good idea in almost any case, too. Similar arguments apply to them, too.

  • $\begingroup$ Thanks for the response! From the probabilities, I made a new column that indicated known or unknown (when prob < 0.7). From there on I changed any predicted behaviour with this low probability into "unknown". Now when I look at the data and make my graphs, a part is classified as unknown :) --- One question though: what do these probabilities actually mean? How were they derived? $\endgroup$
    – Nina
    Oct 31, 2017 at 21:46
  • $\begingroup$ To your last question: look at any tutorial about classification trees, that will be helpful. The leaves of the tree will contain a mixture of classes. If the leaf that a particular future predictor set leads to contains 30% class A, 60% B and 10% C, then the output probabilities will be 30/60/10%. $\endgroup$ Nov 2, 2017 at 9:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.