0
$\begingroup$

@Dear People,

i used firstly the function train() from caret package, to construct-train a classifier with random forests on a merged microarray dataset regarding selected genes, for a binary outcome(cancer vs control), in order to predict cancer samples. Firstly i used 10-fold cross validation with train(), to find the optimum number of mtry and to access some performance measures, which generally "didnt look bad" (trainROC=0.88, TrainSens=0.86, TrainSpec=0.83-with twoClassSummary). Then, based on a tutorial (https://www.biostars.org/p/86981/), i used the "optimum value" mtry returned from train with the randomForest() function with the similar arguments i used on train(same number of ntrees etc). So, my main question is, the output of the randomForest() mentions that "OOB estimate of error rate: 15%"(which altough im not an expert is not too small but maybe not too big). Thus, when i similarly used the code from the above to create a ROC curve plot(AUC), it shows very poor results: AUC=0.14556 !! Is this possible when the specificity and sensitivity have values 83.3 and 86.6 ?

Please excuse me for my naive and maybe "silly" questions, but im a newbie in machine learning and i would like to interpret results and approaches with the most appropriate way !!

$\endgroup$
  • $\begingroup$ Providing a reproducible example will help get your question answered. $\endgroup$ – topepo Nov 3 '15 at 22:15
  • $\begingroup$ Dear Topedo, i just saw your answer ! I will return with the necessary code and example $\endgroup$ – Jason Nov 10 '15 at 8:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.