Questions tagged [sensitivity-specificity]
Sensitivity & specificity are descriptive statistics that measure the quality of a classification system. They are the proportion of real positives that are classified correctly (sensitivity) & the proportion of real negatives that are classified correctly (specificity). They are an integral part of ROC analysis.
216
questions
3
votes
2
answers
109
views
Calculate area under precision-recall curve from area under ROC curve and the prevalence
I am reading material that reports the area under a ROC curve. I am curious to know what the performance would be in precision-recall space. From the sensitivity and specificity values in the ROC ...
- 58.1k
0
votes
0
answers
7
views
Different sensitivity and specificity for balanced populations
I am building a simple test to flag prehistorical hand prints as belonging men or women.
Using a random variable built from the fingers lengths, and having measured a population, I built the ROC curve ...
- 153
0
votes
0
answers
16
views
Optimizing Threshold Selection for Improved Sensitivity in Classification Method Without Validated data
Assume we have a datasheet X, this datasheet contains many of samples with different gorup like G1, ...
- 103
0
votes
0
answers
28
views
What should I do when a test specificity is zero?
Upon doing a test that compares the effect of imaging versus regular tissue biopsy for cancer that has a prevalence of 90%, we ...
- 211
0
votes
0
answers
20
views
What is the relationship between bias-variance and sensitivity-specificity for novelty detection?
An over or under-parameterized binary classification model (- vs +) tends to over or under-fit (bias-variance tradeoff). This leads to errors during prediction on unseen data. Depending on if ...
- 53
1
vote
1
answer
15
views
Is there an equivalent for Yates' correction for a confusion matrix-derived metrics?
Given the following table of predictions vs. actual states:
...
- 1,170
2
votes
1
answer
51
views
Binary classification metrics - Combining sensitivity and specificity?
The harmonic mean between precision and recall (F1 score) is a common metric to evaluate binary classification. It is useful because it strikes a balance between precision (FP) and recall (FN).
For ...
- 1,167
0
votes
0
answers
32
views
Probit "Postestimation", in Stata: How do i specify/utilize results of my probit "Classification tables ("estat classification" command)?
Is there a way or a specific command I can use in Stata for more insights and information after i Run the "estat classification" command and classification Tables for my probit model?
I know ...
- 1
4
votes
1
answer
100
views
What is $\frac{a}{b}$ called in a $y=a x+b$ regression in the context of a physical detection test?
Let's say we have 10 calibrated reference samples of chemical products with a known concentration $x_i$ of a certain chemical component A.
$x_i$ is different for each sample.
We are building a ...
- 458
2
votes
0
answers
28
views
How to do a sensitivity analysis for a Bayesian Model? Prior or Posterior?
I'm developing a mathematical (SIR-style) model for cholera transmission in the Bengal Delta which is fit to data using MCMC.
The purpose of the model is more explanatory rather than predictive, and I ...
2
votes
0
answers
79
views
Can Accuracy be higher than both sensitivity and specificity?
I came across a paper which reported the following results
Accuracy
Specificity
Sensitivity
97.49%
93.6%
94.3%
It seems unusual for accuracy to be higher than both sensitivity and specificity. Is ...
- 131
2
votes
2
answers
67
views
How to obtain positive predictive values from multinomial regression?
For a 2x2 scenario, to determine misclassification we can obtain positive and negative predictive values from a logistic regression model where the outcome is the true value and the exposure is the ...
- 21
1
vote
2
answers
88
views
Why don't we use the harmonic mean of sensitivity and specificity?
There is this question on the F-1 score, asking why we compute the harmonic mean of precision and recall rather than its arithmetic mean. There were good arguments in the answers in favor of the ...
- 211
0
votes
0
answers
96
views
Comparing Sensitivity, Specificity, PPV and NPV of one test across two subgroups
I am seeking to compare the diagnostic performance of a test within a population with two defined subgroups.
I have the raw data & computed data, but am unsure what the best test is to compare the ...
- 1
3
votes
1
answer
752
views
pROC package - sensitivity and specificity calculations
I am using the pROC package in R to generate ROC curves. Using the "coords" function, I can extract the sensitivity (Se) , specificity (Sp), negative predicted value (NPV) and positive ...
- 139
0
votes
0
answers
25
views
ROC analysis with a fuzzy reference standard with estimates of its reliability
I‘m conducting ROC analyses in order to assess the diagnostic accuracy (AUC, sensitivity and specificity for certain cut-offs) of multiple index tests (ordinal scaled measures). The goal is to compare ...
2
votes
1
answer
61
views
AUC - different interpretations
When browsing through literature about ROC - AUC, there seems to be a disparity.
While some plot TPR and FPR, e.g. from Wikipedia: "The ROC curve is created by plotting the true positive rate (...
- 33
0
votes
0
answers
14
views
sensitivity-specificity and other test
I have two different test done in two different facilities. I want to measure the accuracy of each and compare.
I have measured the sensibility, specificity, PPV and NNP of each test using a 2x2 table....
- 1
0
votes
1
answer
238
views
How to select a model based on ROC AUC, sensitivity and specificity?
I'm running several machine learning algorithms on a dataset with 80% negatives and 20% positive cases (classification). Below I attach the results of comparing performance on 500 bootstrap resamples ...
- 13
1
vote
2
answers
286
views
(Why) Does positive predictive value depend on sensitivity?
Assume there are two possible categories, coded as $0$ and $1$.
Define positive predictive value (PPV) to be the probability that an observation really is in category $1$ when it is predicted as being ...
- 58.1k
5
votes
2
answers
307
views
Calculating the Brier or log score from the confusion matrix, or from accuracy, sensitivity, specificity, F1 score etc
Suppose I have a confusion matrix, or alternatively any one or more of accuracy, sensitivity, specificity, recall, F1 score or friends for a binary classification problem.
How can I calculate the ...
- 117k
1
vote
1
answer
223
views
Comparing diagnostic test in different populations
I have 5 diagnostic tests (test A, B, C and D) for a certain disease, tested in 2 different populations.
For comparing test parameters (sensitivity, specificity) of test A and B in the same population ...
16
votes
2
answers
2k
views
Academic reference on the drawbacks of accuracy, F1 score, sensitivity and/or specificity
Accuracy, as a KPI for assessing binary classification models, has major drawbacks: Why is accuracy not the best measure for assessing classification models?. The exact same issues also plague the F1 ...
- 117k
1
vote
0
answers
14
views
Best way to sum up and visualize the relation between results of 2 tests measuring the same outcome with 3 results
I have a dataset of different individuals who underwent 2 test that measure the same outcome with 3 possible results
not reduced
mildly reduced
highly reduced
I know that the distribution of the ...
- 11
0
votes
1
answer
56
views
How can Mass Spectrometry have high sensitivity but average reproducibility?
I am reading up on mass spec from this link: https://www.ebi.ac.uk/training/online/courses/metabolomics-introduction/designing-a-metabolomics-study/comparison-of-nmr-and-ms/
They state:
The two most ...
- 367
1
vote
0
answers
31
views
Sensitivity and specificity for multiple raters
I am conducting a study where 3 raters assessed performance of two similar test in regards to classify a lesion as malignant vs. benign. I wish to calculate specificity and sensitivity of each test ...
- 11
0
votes
0
answers
44
views
How to apply Bayes' theorem on a contingency in which the sampled distribution differs drastically from the entire population?
Summary: How can I calculate my probability of being truly positive based on a contingency table that has a completely different distribution than the total population?
I was recently confronted with ...
- 1
0
votes
1
answer
286
views
Intersection over union vs sensitivity
In the context of segmentation, what is the difference between IoU and sensitivity? It sounds to me like they describe the same formula in different contexts but I might be wrong.
When true ...
- 101
0
votes
0
answers
56
views
Automatically obtain optimum cutoff point for accuracy, sensitivity and specificity
Below is a code snippet to plot the accuracy, sensitivity and specificity. I can manually eyeball and see the cutoff point at approximately around 0.3. I was wondering if there anyway to print out ...
1
vote
2
answers
1k
views
Precision vs. specificity
I know that if we cannot afford to have false positive results, we should aim for high precision.
My question is, how is precision different from specificity?
Any examples?
- 21
2
votes
2
answers
335
views
Sensitivity vs. specificity vs. recall
Given a binary confusion matrix with true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN), what are the formulas for sensitivity, specificity, and recall?
I'm ...
- 219
1
vote
1
answer
57
views
What is the probability of at least one person in a sample of k people being infected?
Given the prevalence, sensitivity and specificity of a single test are known.
- 111
0
votes
0
answers
22
views
Sensitivity to settings vs statistical sensitivity
I am confused between two different properties of a model. Both are called sensitivity. First, sensitivity is another word for true positive rate. Second, the output of a model changes in response to ...
- 87
3
votes
1
answer
111
views
Is a Sensitivity-Specificity curve equal to a horizontally flipped ROC?
So I need to plot a Sensitivity-Specificity curve. Since ROCs represent TPR (sensitivity) against FPR (or 1 - Specificity), can I just plot TPR against 1 - FPR as in the code below to obtain a Sens-...
- 31
1
vote
1
answer
49
views
Why account for censoring in estimating sensitivity but not specificity?
I am reading Tutz & Schmid "Modeling Discrete Time-to-Event Data" (2016) chapter 4 Evaluation and Model Choice section 4.3.3 Discrimination Measures. On p. 94 estimators for sensitivity ...
- 64.6k
2
votes
2
answers
82
views
Calculating "accuracy", "recall" etc. without classification
I have a set of models, that I'm comparing to each other with respect to prediction of a binary event. I'm using a few proper scores (Brier, log), but I also need accuracy, recall, sensitivity etc., ...
- 1,085
2
votes
2
answers
47
views
Using a ROC Plot to interpret specific scores
I have a binary classifier which outputs a given score to differentiate normal (low score) from abnormal (high score) cases. The score itself however is non-interpretable to others.
I know a ROC plot ...
- 1,476
1
vote
2
answers
55
views
Individual Measurement in Overlapping Distributions
I have a situation where I have measurements taken from samples of healthy and pathological populations. The pathological populations tend to have much higher measurements, but the distributions are ...
- 1,476
1
vote
1
answer
32
views
How to show that simple random sample sensitivity is unbiased for population sensitivity
In diagnostic testing, sensitivity $S$ is the probability that the test gives a positive result given that you have the condition being tested. From a simple random sample of people who take the test, ...
0
votes
1
answer
168
views
Confidence intervals for series of Covid tests
Suppose I take two antigen tests and each test has a specificity of 75.3% (95% confidence interval from 65.8% to 83.4%) e.g. from https://pubmed.ncbi.nlm.nih.gov/33455451/
A. what is the sensitivity ...
- 1
0
votes
0
answers
38
views
Post test probability with a ROC curve
I have data that is normally distributed related to risk of a particular disease. At the median of the distribution, you would expect to observe the population prevalence level of disease P0=0.01. For ...
- 1
0
votes
0
answers
221
views
Calculating true positive and true negative from sensitivity and specificity
I know the specificity and sensitivity of a test are both 90%. The incidence is 2.5%.
The total population is 10,000 and the number of people with the illness is 250, how do I calculate the true ...
- 1
2
votes
2
answers
694
views
Logistic regression metric
I am interested to understand in which scenarios person should use sensitivity, specificity, and when should person opt for precision recall.
On a high level I understand for a balanced data set we ...
- 21
1
vote
1
answer
80
views
Is Average Accuracy An Ill-Defined Metric? [closed]
Five people each took a 100 question yes/no question test. I have their individual accuracy scores and was planning to report their average.
However, I was informed that "You cannot take an ...
- 33
2
votes
1
answer
467
views
Create AUC-ROC from single sensitivity and specificity value? [duplicate]
Is it possible and appropriate to estimate the area under the receiver operating characteristic curve from a single point estimate of an individual's sensitivity and specificity performance?
- 33
0
votes
0
answers
20
views
Roc curve straight line interpretation [duplicate]
How can I fix the straight line? is that mean that 70% of my cases that tag as positive and actually negative have the same value?
1
vote
1
answer
300
views
Accuracy, Recall (sensitivity), Specificity confusion
I came up with this paper with 765 citations!
On page 2, it expresses an equation relating these metrics:
Accuracy
sensitivity (a.k.a Recall).
specificity (a.k.a 1 - type1 error rate)
prevalence (...
- 455
0
votes
0
answers
153
views
S-Shaped/Vertical ROC curve crossing the diagonal
I am running binary logistic regression and when the ROC curve is created this is the output (see below):
I believe it makes sense somewhat, as the model does have 100% specificity and 0% sensitivity ...
3
votes
1
answer
153
views
Sensitivity / specificity with dependent observations
Suppose we have a classification scenario, where each observation is comprised from a reader, a case and the status (0 or 1). Each reader get several cases (but each case is shown to only one reader). ...
- 1,323
2
votes
5
answers
108
views
Does a term exist for this concept in statistics?
Suppose you have a dataset where the covariates are biomedical information for a given patient (e.g. height, weight, blood type, etc.) and the response variable is whether the patient has a certain ...
