I want to know how to calculate AUC to compare correlation methods.

I read this paper http://www.ncbi.nlm.nih.gov/pubmed/23962479

Is there any idea how the authors of above paper have calculated AUC for each method?

UPDATE: Here is the Full Paper

According to authors;

To calculate the area under the ROC curve, we computed the Riemman sum with intervals of 0.001.

Table 1 describes the areas under ROC curves enter image description here

  • $\begingroup$ AUC is used for methods that enable prediction, while correlation describes strength of relation between two variables, so it is not really clear what you are asking..? The paper you refer to is not available in open access, so we are not possible to refer to it for additional hints. $\endgroup$ – Tim May 8 '15 at 15:48
  • $\begingroup$ I have updated the question. I think that authors have calculated AUC. $\endgroup$ – statuser May 9 '15 at 19:33
  • 1
    $\begingroup$ We do not have access to this article since it is not in open access so it is impossible for us to answer. As @AdamO said, AUC is used for measuring classifier accuracy while correlation is not used for classifying but rather for describing relations in the data. Until you provide a detailed summary of the paper you refer to, so that we know what was described there, we cannot answer. $\endgroup$ – Tim May 11 '15 at 6:30
  • $\begingroup$ After reading the paper my guess is that they used $p$-values for those measures as a cutoff as a prediction criteria for existence of a relation, and then used a classical ROC/AUC for binary classifiers. $\endgroup$ – Tim May 11 '15 at 9:30
  • $\begingroup$ What about Reimann sum then? $\endgroup$ – statuser May 11 '15 at 9:41

The paper does not use AUC at all. And rightly so. The Area Under the (receiver operator characteristic) Curve (AUC) is a measure of binary classification accuracy. The paper is focused on continuous outcomes. Mainly gene expression.

In a contrived sense, the AUC is equivalent to another association measure: the rank based U-statistic. This measures correlation in a probabilistic sense; it is for any two randomly sampled observations, $(X_1, Y_1)$ and $(X_2, Y_2)$ the probability $P(X_2 > X_1 | Y_2 > Y_1)$. However, the U-statistic is a weak measure of correlation at best. It's not really appropriate for genomics where actually modeling the exact functional form of the relationship is often more important.

  • $\begingroup$ Check updated table. $\endgroup$ – statuser May 9 '15 at 19:34
  • 2
    $\begingroup$ @statuser yes but what are they classifying that they calculate an AUC for? $\endgroup$ – AdamO May 11 '15 at 4:57
  • $\begingroup$ They are evaluating the power of test by using ROC curve. "The area under the ROC curve is a quantitative summary of the power of the employed test and it varies from 0 to 1. In other words, an area close to 1 denotes high power, whereas an area below 0.50 means that the method is not able to identify dependence. An area close to 0.50 is equivalent to random decisions. To calculate the area under the ROC curve, we computed the Riemman sum with intervals of 0.001." $\endgroup$ – statuser May 11 '15 at 9:48
  • $\begingroup$ @statuser this is not a valid method for assessing statistical tests. The reason why is that statistical testing never classifies a "null finding" as such. It just says a finding is inconsistent with a null hypothesis or not. If you are assessing a statistical test, it is for calibration and power and you would use a qqplot or power curve to evaluate this. $\endgroup$ – AdamO May 11 '15 at 14:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.