All Questions
Tagged with model-evaluation sensitivity-specificity
10 questions
12
votes
3
answers
3k
views
Classifier performance measure that combines sensitivity and specificity?
I have 2-classes labelled data on which I'm performing classification using multiple classifiers. And the datasets are well balanced. When assessing the classifiers' performance, I need to take into ...
- 355
5
votes
4
answers
5k
views
Precision vs. specificity
I know that if we cannot afford to have false positive results, we should aim for high precision.
My question is, how is precision different from specificity?
Any examples?
- 71
3
votes
1
answer
82
views
AUC - different interpretations
When browsing through literature about ROC - AUC, there seems to be a disparity.
While some plot TPR and FPR, e.g. from Wikipedia: "The ROC curve is created by plotting the true positive rate (...
- 43
3
votes
1
answer
2k
views
Sensitivity and specificity for non-binary outcome
I'm very new to machine learning (this is my second project and only the first significant one) and here's the problem I'm currently dealing with:
I am evaluating the performance of a model that ...
user136151
3
votes
2
answers
108
views
Calculating "accuracy", "recall" etc. without classification
I have a set of models, that I'm comparing to each other with respect to prediction of a binary event. I'm using a few proper scores (Brier, log), but I also need accuracy, recall, sensitivity etc., ...
- 1,127
3
votes
0
answers
176
views
Assessing correlated predictions
Let's assume we have a prediction algorithm (if it helps, imagine it's using some boosted tree method) that does daily predictions for whether some event will happen to a unit (e.g. a machine that ...
- 35.2k
2
votes
3
answers
1k
views
Calculate the confidence interval of a balanced accuracy by taking the mean of the CIs of sensitivity and specificity?
Because sensitivity and specificity are typically estimated as binomial proportions (e.g. k = TP, n = TP+FN), we can use any of the methods used to estimate the confidence interval for binomial ...
- 41
1
vote
1
answer
2k
views
Is it possible for a model to have higher sensitivity/specificity but lower accuracy and AUC?
In the evaluation of classification models, I've found one model to have a higher accuracy and c-statistic (AUC) as compared to a second model. However, the second model has higher sensitivity, ...
- 163
1
vote
0
answers
88
views
How to make specificity a stable evaluation metric?
ROC (including AUC) metrics are widely used for binary classification problems. AUC is usually selected to evaluate the model.
However, some tasks may require high specificity with fixed sensitivity. ...
- 439
0
votes
0
answers
139
views
Can AUC increase while both sensitivity and specificity decrease?
Assume I am given two classifiers - A and B. Is it possible that the specificity and sensitivity obtained from using B will be higher than A, but the AUC of A will be higher than the AUC of B?
Will ...
- 1