I have a dataframe, 'datas', with 200 observations and a series of columns (some numeric, dummy, etc) and a binary class variable to be predicted that is called "bad_econ." I would like to get the model to predict whether bad_econ = is the case (1) or not (0). Simple enough.
I wrote the follow code, testing rpart (I also tried Tree which yielded different model but same bad results afterwards):
frmla <- bad_econ ~ blah1 + blah2 + blah3 + [etc...] fit = rpart(frmla, method='class', data=datas) printcp(fit) # display the results plotcp(fit) # visualize cross-validation results summary(fit) # detailed summary of splits fit
This works, and outputs some results but I am used to dealing with AUC + confusion matrix to assess model validity, not "residual mean deviance" nor "distribution of residuals" which are totally foreign to me, despite research.
So, I write the following, feeding my original data back through the model:
pred = predict(fit) table(pred, datas$bad_econ)
I would think this would yield a small table that looks like this:
pred 0 1 0 x y 1 z a
But instead I get the following...
pred 0 1 0 8 0 0.2 4 1 0.333333333333333 8 4 0.4 6 4 0.666666666666667 5 10 0.714285714285714 2 5 0.782608695652174 5 18 0.8 1 4 0.857142857142857 1 6 0.928571428571429 1 13 1 0 93
Why is it passing so many fractional values when all the values in that column are binary and I've forced method = "class" on the model AND type="class" on the predict?
So, (a) what is going wrong here, (b) does anyone have any recommendations on how I can get what I'm actually looking for out of this (confusion matrix, AUC) and/or does anyone have an explanation of residual mean deviance that doesn't look like I need a M.S in Stats to swallow it?
e.g. How effective was this?
Residual mean deviance: 0.09359 = 17.13 / 183 Distribution of residuals: Min. 1st Qu. Median Mean 3rd Qu. Max. -0.9286 0.0000 0.0000 0.0000 0.1429 0.8000
Sorry this is lengthy and thank you for any responses - looking forward to being more involved in the community on some things I actually have done before :)