rpart classification: why is my predict() output not adhering to type="class"?

Question

I have a dataframe, 'datas', with 200 observations and a series of columns (some numeric, dummy, etc) and a binary class variable to be predicted that is called "bad_econ." I would like to get the model to predict whether bad_econ = is the case (1) or not (0). Simple enough.

I wrote the follow code, testing rpart (I also tried Tree which yielded different model but same bad results afterwards):

frmla <- bad_econ ~ blah1 + blah2 + blah3 + [etc...]

fit = rpart(frmla, method='class', data=datas)

printcp(fit) # display the results
plotcp(fit) # visualize cross-validation results
summary(fit) # detailed summary of splits
fit

This works, and outputs some results but I am used to dealing with AUC + confusion matrix to assess model validity, not "residual mean deviance" nor "distribution of residuals" which are totally foreign to me, despite research.

So, I write the following, feeding my original data back through the model:

pred = predict(fit)
table(pred, datas$bad_econ)

I would think this would yield a small table that looks like this:

pred 0  1
  0  x  y
  1  z  a

But instead I get the following...

pred                 0  1
  0                  8  0
  0.2                4  1
  0.333333333333333  8  4
  0.4                6  4
  0.666666666666667  5 10
  0.714285714285714  2  5
  0.782608695652174  5 18
  0.8                1  4
  0.857142857142857  1  6
  0.928571428571429  1 13
  1                  0 93

Why is it passing so many fractional values when all the values in that column are binary and I've forced method = "class" on the model AND type="class" on the predict?

So, (a) what is going wrong here, (b) does anyone have any recommendations on how I can get what I'm actually looking for out of this (confusion matrix, AUC) and/or does anyone have an explanation of residual mean deviance that doesn't look like I need a M.S in Stats to swallow it?

e.g. How effective was this?

Residual mean deviance:  0.09359 = 17.13 / 183 
Distribution of residuals:
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
-0.9286  0.0000  0.0000  0.0000  0.1429  0.8000

Sorry this is lengthy and thank you for any responses - looking forward to being more involved in the community on some things I actually have done before :)

Answer 1

Predicting with an rpart object also includes the parameter Type. From the help:

If the rpart object is a classification tree, then the default is to return prob predictions, a matrix whose columns are the probability of the first, second, etc. class.

In your case it returns the probabilities of the classes, not the class itself. Use the following code to get the classes.

pred = predict(fit, type = "class")
table(pred, datas$bad_econ)

for more information on residual mean deviance check this post on cv

Edit: For measuring AUC and plot the ROC, you could use these commands:

library(ROCR)
rocrpred <- prediction(pred[,2], data$bad_econ)
# print AUC value
as.numeric(performance(rocrpred, "auc")@y.values)
# plots the ROC curve with colors where the splits are.
plot(performance(rocrpred, "tpr", "fpr"), colorize = TRUE) 

But read the help for the performance function. Because it can also create lift charts, Sensitivity/specificity plots, or Precision/recall graphs.

I very seldom look at the residual mean deviance. But I compare the confusion matrix of the prediction model with the confusion matrix of a baseline model. Same goes for the roc curve and the auc values. Those are easier to explain to non-stats people.