0
$\begingroup$

I am using a random forest classification to compare how the classification of disease improves when combining metabolites with two other measures ( visceral fat and CRP-1 levels) to see if adding the metabolite information improves classification than just visceral fat and CRP-1 levels together.

enter image description here

Adding metabolites only improves classification slightly ( I have small sample numbers and a small number of metabolites which doesn't help).

Here is the output from metabolites, visceral fat and CRP-1 levels enter image description here

Here is the output from just the visceral fat and CRP levels enter image description here

Whilst from this tutorial using the iris data, three lines are present on the ROC ( one for each species). https://www.blopig.com/blog/2017/04/a-very-basic-introduction-to-random-forests-using-r/

Please could anybody tell me how I would be able to plot two lines for the ROC curve for the disease group ( false positive vs true positive) showing one with the metabolites + visceral fat + CRP-1, whilst another with just visceral fat + CRP-1 ( a bit like the image above but the graph will only have two lines). I cannot seem to find any example online at the moment. My outcomes are not just from one model as with the iris data, as I ran two different ones ( one with the two variables, the other with the two variables + metabolites).

Any help or link to an example would be very appreciated.

$\endgroup$
1
  • $\begingroup$ I think the main problem here is that you are having $55$ points in your OOB predictions. Use repeated resampling (repeated $k$-fold, stratified bootstrap, pick your favourite) to get more stable estimates. $\endgroup$
    – usεr11852
    May 22, 2020 at 16:22

1 Answer 1

0
$\begingroup$

You can use pROC to add the two lines on to one plot

  ###model with metabolites, visceral fat and CRP-1
 result.predicted.prob <- predict(rf.fit, mod_test.newy, type="prob") 

  result.roc <- roc(mod_test.newy$Disease, result.predicted.prob$UC)



 ##model with just visceral fat and CRP-1
 result.predicted.prob2 <- predict(rf.fit2, justviscfat.crp.test, type="prob") 

 result.roc2 <- roc(justviscfat.crp.test$Disease, result.predicted.prob2$UC)

From the CRAN pages for pROC https://cran.r-project.org/web/packages/pROC/pROC.pdf

  # We need a plot to be ready
  plot(result.roc, type = "n") # but don't actually plot the curve
  # Add the line
  lines(result.roc, type="b", pch=21, col="cadetblue", bg="grey") 
  # Add the line of an other ROC curve
  lines(result.roc2, type="o", pch=19, col="chocolate")  
  legend("bottomright", legend=c("Metabolites + Visceral Fat + CRP-1", "Visceral Fat + 
  CRP-1") ,col=c("cadetblue", "chocolate"), lwd=2)

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.