# How do I display my output comparing the effect of variables on classification of disease from a random forest analysis in R using an AUROC?

I am using a random forest classification to compare how the classification of disease improves when combining metabolites with two other measures ( visceral fat and CRP-1 levels) to see if adding the metabolite information improves classification than just visceral fat and CRP-1 levels together.

Adding metabolites only improves classification slightly ( I have small sample numbers and a small number of metabolites which doesn't help).

Here is the output from metabolites, visceral fat and CRP-1 levels

Here is the output from just the visceral fat and CRP levels

Whilst from this tutorial using the iris data, three lines are present on the ROC ( one for each species). https://www.blopig.com/blog/2017/04/a-very-basic-introduction-to-random-forests-using-r/

Please could anybody tell me how I would be able to plot two lines for the ROC curve for the disease group ( false positive vs true positive) showing one with the metabolites + visceral fat + CRP-1, whilst another with just visceral fat + CRP-1 ( a bit like the image above but the graph will only have two lines). I cannot seem to find any example online at the moment. My outcomes are not just from one model as with the iris data, as I ran two different ones ( one with the two variables, the other with the two variables + metabolites).

Any help or link to an example would be very appreciated.

• I think the main problem here is that you are having $55$ points in your OOB predictions. Use repeated resampling (repeated $k$-fold, stratified bootstrap, pick your favourite) to get more stable estimates. May 22, 2020 at 16:22

You can use pROC to add the two lines on to one plot

  ###model with metabolites, visceral fat and CRP-1
result.predicted.prob <- predict(rf.fit, mod_test.newy, type="prob")

result.roc <- roc(mod_test.newy$$Disease, result.predicted.prob$$UC)

##model with just visceral fat and CRP-1
result.predicted.prob2 <- predict(rf.fit2, justviscfat.crp.test, type="prob")

result.roc2 <- roc(justviscfat.crp.test$$Disease, result.predicted.prob2$$UC)


From the CRAN pages for pROC https://cran.r-project.org/web/packages/pROC/pROC.pdf

  # We need a plot to be ready
plot(result.roc, type = "n") # but don't actually plot the curve
# Add the line
lines(result.roc, type="b", pch=21, col="cadetblue", bg="grey")
# Add the line of an other ROC curve
lines(result.roc2, type="o", pch=19, col="chocolate")
legend("bottomright", legend=c("Metabolites + Visceral Fat + CRP-1", "Visceral Fat +
CRP-1") ,col=c("cadetblue", "chocolate"), lwd=2)