1
$\begingroup$

I have a microarray data which I ran through a continuous covariate (say "X"). I did this using 4 different methods.

For the results obtained from each of the 4 methods, I have the following from each of the 4:

  1. p-value,
  2. FDR significant value,
  3. "Bonferonni significance" (saying "true" or "false")
  4. "Holm significance" (saying "true" or "false")

I wish to obtain an ROC curve showing lines for sensitivity and specificity from each of the 4 methods.

I kind of know what is an ROC curve, but even after reading through a few links, I dont feel clear of the concept of what kind of input it needs to create it. So I apologise if this seems to be a dummy question.

So my questions are:

  1. Can a ROC curve be created from the 4 values (p value, FDR, Bonferoni & Holm) which I have mentioned above? Or is it that I have to calculate the FP and FN first?

  2. Can you suggest me an easy to use package which could do this for me, by giving me a ROC curve with lines of 4 methods using the input above?

Your help appreciated. Thanking you in advance.

$\endgroup$
  • $\begingroup$ you need to be able to compute FP/FN that depends on a threshold (say a p-value), then just plot ROC by changing the threshold. If you only need the area under the ROC, take a look at Wilcoxon rank-sum test. $\endgroup$ – Memming Feb 5 '13 at 14:07
  • $\begingroup$ Welcome to cross validated! As your question is on the proper site now, have a look at the listed "Related" questions. Do they answer your question? You may also want to check the warnings about model comparison on the basis of ROC that have been written here many times. $\endgroup$ – cbeleites supports Monica Feb 5 '13 at 14:56
1
$\begingroup$

ROC curve exists only when you have a binary true decision and a continuous prediction score that an object belongs to one of those two classes.

So, ROC is possible only for p-value and FDR; for Bonferonni and Holm use some binary prediction accuracy measure like precision, recall, F-score, accuracy, whatever.

As for an R package, try ROCR; if you only want AUROC or just something simple (RORC may feel overwhelming and is in general pretty slow) yet less functional, use colAUC from caTools.

$\endgroup$
  • $\begingroup$ Thank you for your good answer @mbq. I kind of now get what you say. I have now a specific question for my above question. Lets say I consider only one method. I wish to calculate ROC for the same. I have also come across this package called "pROC" given on this link (last figure for statistical comparison). In my case, can I give my input as p-value and FDR only? $\endgroup$ – Letin Feb 5 '13 at 15:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.