12
$\begingroup$

Normally we cannot draw an ROC curve for the discrete classifiers like decision trees. Am I right? Is there any way to draw an ROC curve for Dtrees?

$\endgroup$
  • 1
    $\begingroup$ You can find your aswer here: stats.stackexchange.com/questions/105501/… $\endgroup$ – rapaio Jul 4 '14 at 6:41
  • $\begingroup$ [Decision tree model evaluation for “training set ” vs “testing set ” in R][1] [1]: stats.stackexchange.com/questions/49416/… You can find your answer here. $\endgroup$ – Sangram Jul 4 '14 at 6:57
  • $\begingroup$ @rapaio Sorry your link shows a ROC curve to find a threshold in a classifier which produce output between 1 and 0 (continuous value). $\endgroup$ – DataMiner Jul 4 '14 at 7:09
  • $\begingroup$ @Sangram These are the solutions in R but i want to just understand how it works. $\endgroup$ – DataMiner Jul 4 '14 at 7:11
15
$\begingroup$

If your classifier produces only factor outcomes (only labels), without scores, you still can draw a ROC curve. However this ROC curve is only a point. Considering the ROC space, this points is $(x,y) = (\text{FPR}, \text{TPR})$, where $\text{FPR}$ - false positive rate and $\text{TPR}$ - true positive rate.

See more on how this is computed on Wikipedia page.

You can extend this point to look like a ROC curve by drawing a line from $(0,0)$ to your point, and from there to $(1,1)$. Thus you have a curve.

However, for a decision tree is easy to extend from an label output to a numeric output. Note that when you predict with a decision tree you go down from the root node to a leaf node, where you predict with majority class. If instead of that class you would return the proportion of classes in that leaf node, you would have a score for each class. Suppose that you have two classes $\text{T}$ and $\text{F}$, and in your leaf node you have 10 instances with $\text{T}$ and 5 instances with $\text{F}$, you can return a vector of scores: $(\text{score}_T, \text{score}_F) = ( \frac{\text{count}_T}{\text{count}_T + \text{count}_F}, \frac{\text{count}_F}{\text{count}_T + \text{count}_F}) = (10/15, 5/15) = (0.66, 0.33)$. Take care that this is really note a proper scoring rule (this are not the best estimator for probabilities), but is better than nothing I believe, and this is how usually scores are retrieved for decision trees.

$\endgroup$
12
$\begingroup$

For a Decision Tree, the classes are still predicted with some level of certainty. The answer is already given by @rapaio, but I'll expand on it a bit.

Imagine the following decision tree (it's a little bit modified version of this one)

decision tree example

At each node there are not only the majority class labels, but also others what ended up at that leaf, so we can assign the degree of certainty to that leaf at which we predict the label.

For example, consider the following data

data

We run it, and assign the scores to the output, not the actual labels. With this, we can draw a ROC curve, like suggested here

roc curve

It makes little sense, though, to use it to tune your threshold (since, of course, there's no such thing as threshold in Decision Trees), but it still can be used to calculate the AUC, which, in this case, is 0.92

R code used here:

outlook = c('rain', 'overcast', 'rain', 'sunny', 'rain', 
            'rain', 'sunny', 'overcast', 'overcast', 'overcast', 
            'sunny', 'sunny', 'rain', 'rain', 'overcast',
            'sunny', 'overcast', 'overcast', 'sunny', 'sunny',
            'sunny', 'overcast')
humidity = c(79, 74, 80, 60, 65, 79, 60, 74, 77, 80, 
             71, 70, 80, 65, 70, 56, 80, 70, 56, 70,
             71, 77)
windy = c(T, T, F, T, F, T, T, T, T, F, T, F, F, F, T, T, F, T, T, F, T, T)
play = c(F, F, T, F, T, F, F, T, T, T, F, F, T, T, T, T, T, T, F, T, F, T)

game = data.frame(outlook, humidity, windy, play)
game$score = NA

attach(game)
game$score[outlook == 'sunny' & humidity <= 70] = 5/8
game$score[outlook == 'sunny' & humidity > 70] = 1 - 3/4
game$score[outlook == 'overcast'] = 4/5
game$score[outlook == 'rain' & windy == T] = 1 - 2/2
game$score[outlook == 'rain' & windy == F] = 3/3
detach(game)

game$predict = game$score >= 0.5
game$correct = game$predict == game$play

library(ROCR)

pred = prediction(game$score, game$play)
roc = performance(pred, measure="tpr", x.measure="fpr")
plot(roc, col="orange", lwd=2) 
lines(x=c(0, 1), y=c(0, 1), col="red", lwd=2)

auc = performance(pred, 'auc')
slot(auc, 'y.values')
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.