Linked Questions

3
votes
1answer
2k views

draw roc curve on an example of 10 probability scores [duplicate]

I'm studying machine learning and find an example question on the book which really confused me. Q: A scoring classifier is evaluated on a test set of 10 examples resulting in the following ...
1
vote
0answers
792 views

Area under the ROC curve for continuous variable [duplicate]

I have the true output as the ratio between 0 and 1. I am trying predict the output using regression. I am supposed to find the area under the ROC curve between the prediction and true values. I am ...
0
votes
1answer
264 views

How are AUROC scores computed with just two vectors of actual and predicted values as input? [duplicate]

In the R package ModelMetrics, the auc score as shown in the documentation takes only two inputs; aucScore <- auc(actual=actuallabels, predicted=predictedlabels) where the inputs are pretty self ...
0
votes
0answers
118 views

ROC curve interpretation [duplicate]

In the context of binary classification how do you interpret ROC curve: more precisely: 1) Why the diagonal stand for a random classifier? [Edit] Let's imagine a random classifier: each time he ...
0
votes
0answers
64 views

How to plot ROC for LOOCV by hand based on output probabilities returned by SVM? [duplicate]

I want to plot ROC for Leave-one-out (LOO) cross-validation (CV). I have the decision values as well as probability values for each class from SVM classifier. I did see the answer to this post; ...
1
vote
0answers
60 views

Calculating AUC from continuous output [duplicate]

Calculating AUC for one threshold in the continuous output is simple: AUC = (TPR - FPR + 1) / 2; What if I want to calculate AUC for multiple thresholds in ...
0
votes
0answers
56 views

How to interpret ROC curve? [duplicate]

I am currently doing a classification problem for classifying the functional class and non-functional class of peptidase cleavage site. The data on non-functional class (negative class) is highly ...
229
votes
5answers
341k views

What does AUC stand for and what is it?

Searched high and low and have not been able to find out what AUC, as in related to prediction, stands for or means.
57
votes
1answer
57k views

Understanding ROC curve

I'm having trouble understanding the ROC curve. Is there any advantage / improvement in area under the ROC curve if I build different models from each unique subset of the training set and use it to ...
27
votes
3answers
26k views

Why is AUC higher for a classifier that is less accurate than for one that is more accurate?

I have two classifiers A: naive Bayesian network B: tree (singly-connected) Bayesian network In terms of accuracy and other measures, A performs comparatively worse than B. However, when I use the R ...
4
votes
2answers
3k views

How does AUC of ROC equal concordance probability? [duplicate]

If this answer on Quora is correct then I think I understand what concordance probability is. However, I also find that this answer on StackExchange provides the formula of concordance probability ...
3
votes
1answer
3k views

How to compare predictive power of survival models?

Frank Harrell describes the concordance (or Somer's D) as not being sensitive enough to compare multiple survival models for their diagnostic ability, and I've observed this in my own work with ...
5
votes
1answer
692 views

How is a ROCAUC=1.0 possible with imperfect accuracy? [duplicate]

I used sklearn to compute roc_auc_score for a dataset of 72 instances. The accuracy was at 97% (2 misclassifications), but the ROC AUC score was 1.0. How is this ...
1
vote
2answers
2k views

Meaning of model calibration

A previous post has discussed model discrimination very nicely. The post also briefly discussed calibration: "When evaluating a risk model, calibration is also very important. To examine this, ...
2
votes
3answers
662 views

AUC with incomplete ROC curve

I am doing an experiments where changing a parameter I am obtaining different number of FalsePositive, FalseNegative... and so on. I am using this parameter tuning as threshold tuning to obtain FPR ...

15 30 50 per page