All Questions
Tagged with specificity or sensitivity-specificity
226 questions
0
votes
2
answers
51
views
Is it possible that false-positive rate decreases with increasing prevalence?
I am interested in the effect of prevalence on prediction performance. Chouldechova (2016) states that:
[w]hen using a test-fair [recidivism prediction instrument] in
populations where recidivism ...
- 113
0
votes
0
answers
22
views
Se, Sp, NPV and PPV questions for repeated measures data
I have a dataset that contains multiple test results (expressed as %) per participant, at various time points post kidney transplant. The dataset also contains the rejection group the participant ...
- 173
1
vote
0
answers
22
views
Hypergeometric approximation of herd sensitivity and its non-linear nature
I have recently been exploring the use of the equation described in Cannon 2001 paper "Sense and Sensitivity" on herd sensitivity (love the name).
I have been looking to test a hypothetical ...
- 63
0
votes
0
answers
27
views
Meta-analysis of ROC areas with missing confidence interval
I am performing a meta-analysis of a diagnostic score going from 0 to 10 based on 30 reports. For each report I have data on number of false/true positive/negative patients for one and in some cases ...
- 301
0
votes
1
answer
32
views
Comparison Sensitivity and Specificity - hold one stable for comparison of the other?
Given two binary diagnostic tests, T1 and T2, I want to compare the Sensitivity and Specificity of the two tests when given to a sample of subjects. Each subject will receive both tests, and there is ...
- 1,170
0
votes
2
answers
43
views
Relationship between sensitiviy, probability of detection and Prevalence
Folks, I have 3 questions:
If I know the number of exams performed with a specific test on a group of patients and the test's sensitivity, can I directly estimate the overall probability of detecting ...
0
votes
1
answer
35
views
Propensity score matching with extremly small data size. Is there any alternative method that works with small data?
I have two groups, A and B, which represent two different imaging methods for detecting a rare disease. This is not a typical treatment-control setup; instead, it involves comparing the efficacy of ...
- 108
1
vote
1
answer
26
views
Assessing the Implementation and Performance of a New Biomarker in Cancer Surveillance
In a surveillance program designed to detect a specific type of cancer in a high-risk population, both biomarker A and ultrasound (US) are used semi-annually. If either A or US returns a positive ...
- 63
0
votes
0
answers
43
views
How to measure Statistical Significance for calculated Sensitivity, Specificity, Precision, Accuracy and f1 values?
I am trying to understand how to calculate one or more measures of statistical significance to display alongside metrics I've calculated from my data.
Abbreviations I am using in the rest of this post:...
- 1
0
votes
0
answers
42
views
How to properly compare a single diagnostic test in two subpopulations
Suppose I have a diagnostic test with a binary response. I have performed this test on some population.
Now, I split this population in two subgroups (e.g. male/female), and would like to check if ...
- 115
0
votes
0
answers
7
views
How to draw a ROC curve given estimated probability that a unit is positive and actual observed class? [duplicate]
Assume that a classification model fitted to data available to you has provided for each statistical unit a probability $P(+|x)$ that the unit is positive. The following table shows all available ...
- 3
0
votes
0
answers
21
views
How can we apply sensitivity analysis in comparing two ARIMA models, both measuring intervention effect?
I have a dataset of reporting rates collected over an 8-month period obtained from the baseline (4 months) and during intervention (4 months). The goal is to determine if the intervention has a ...
0
votes
0
answers
7
views
How to compute the minimum sensitivity and specificity a classifier needs to estimate a proportion to within some percent error?
I am building a binary classifier. Once classified, the proportion of positive events will be calculated. The predicted proportion is required to be within 10% of the true proportion. In practice, ...
- 1
1
vote
0
answers
17
views
Interpreting sobol indices
I am trying to test the sensitivity of my model to a change in the parameters std and translation_factor. I get the following results :
Sobol First Order Indices:
translation_factor: -1....
- 11
1
vote
1
answer
164
views
Issues with metaprop function for computing pooled specificity in meta-analysis
I am training to learn how to perform meta-analysis using R. I conducted a meta-analysis using the metaprop function in R with the provided dataset. The goal is to ...
- 13
6
votes
1
answer
92
views
ROCAUC = average sensitivity across all thresholds according to IEEE TPAMI, yet my calculations show otherwise
Carrington et al (2023) make the claim that area under the receiver-operator characteristic curve is equal to the average sensitivity across all thresholds, and similarly for specificity (section 3), ...
- 67.1k
3
votes
2
answers
198
views
Calculate area under precision-recall curve from area under ROC curve and the prevalence
I am reading material that reports the area under a ROC curve. I am curious to know what the performance would be in precision-recall space. From the sensitivity and specificity values in the ROC ...
- 67.1k
0
votes
0
answers
63
views
What is the relationship between bias-variance and sensitivity-specificity for novelty detection?
An over or under-parameterized binary classification model (- vs +) tends to over or under-fit (bias-variance tradeoff). This leads to errors during prediction on unseen data. Depending on if ...
- 53
1
vote
1
answer
23
views
Is there an equivalent for Yates' correction for a confusion matrix-derived metrics?
Given the following table of predictions vs. actual states:
...
- 1,291
2
votes
1
answer
118
views
Binary classification metrics - Combining sensitivity and specificity?
The harmonic mean between precision and recall (F1 score) is a common metric to evaluate binary classification. It is useful because it strikes a balance between precision (FP) and recall (FN).
For ...
- 1,257
4
votes
1
answer
110
views
What is $\frac{a}{b}$ called in a $y=a x+b$ regression in the context of a physical detection test?
Let's say we have 10 calibrated reference samples of chemical products with a known concentration $x_i$ of a certain chemical component A.
$x_i$ is different for each sample.
We are building a ...
- 622
3
votes
0
answers
42
views
How to do a sensitivity analysis for a Bayesian Model? Prior or Posterior?
I'm developing a mathematical (SIR-style) model for cholera transmission in the Bengal Delta which is fit to data using MCMC.
The purpose of the model is more explanatory rather than predictive, and I ...
5
votes
1
answer
332
views
Can Accuracy be higher than both sensitivity and specificity?
I came across a paper which reported the following results
Accuracy
Specificity
Sensitivity
97.49%
93.6%
94.3%
It seems unusual for accuracy to be higher than both sensitivity and specificity. Is ...
- 163
2
votes
2
answers
101
views
How to obtain positive predictive values from multinomial regression?
For a 2x2 scenario, to determine misclassification we can obtain positive and negative predictive values from a logistic regression model where the outcome is the true value and the exposure is the ...
- 21
1
vote
4
answers
239
views
Why don't we use the harmonic mean of sensitivity and specificity?
There is this question on the F-1 score, asking why we compute the harmonic mean of precision and recall rather than its arithmetic mean. There were good arguments in the answers in favor of the ...
- 211
0
votes
0
answers
225
views
Comparing Sensitivity, Specificity, PPV and NPV of one test across two subgroups
I am seeking to compare the diagnostic performance of a test within a population with two defined subgroups.
I have the raw data & computed data, but am unsure what the best test is to compare the ...
- 1
4
votes
1
answer
2k
views
pROC package - sensitivity and specificity calculations
I am using the pROC package in R to generate ROC curves. Using the "coords" function, I can extract the sensitivity (Se) , specificity (Sp), negative predicted value (NPV) and positive ...
- 199
3
votes
1
answer
82
views
AUC - different interpretations
When browsing through literature about ROC - AUC, there seems to be a disparity.
While some plot TPR and FPR, e.g. from Wikipedia: "The ROC curve is created by plotting the true positive rate (...
- 43
0
votes
1
answer
489
views
How to select a model based on ROC AUC, sensitivity and specificity?
I'm running several machine learning algorithms on a dataset with 80% negatives and 20% positive cases (classification). Below I attach the results of comparing performance on 500 bootstrap resamples ...
- 13
1
vote
2
answers
805
views
(Why) Does positive predictive value depend on sensitivity?
Assume there are two possible categories, coded as $0$ and $1$.
Define positive predictive value (PPV) to be the probability that an observation really is in category $1$ when it is predicted as being ...
- 67.1k
8
votes
2
answers
475
views
Calculating the Brier or log score from the confusion matrix, or from accuracy, sensitivity, specificity, F1 score etc
Suppose I have a confusion matrix, or alternatively any one or more of accuracy, sensitivity, specificity, recall, F1 score or friends for a binary classification problem.
How can I calculate the ...
- 131k
1
vote
1
answer
386
views
Comparing diagnostic test in different populations
I have 5 diagnostic tests (test A, B, C and D) for a certain disease, tested in 2 different populations.
For comparing test parameters (sensitivity, specificity) of test A and B in the same population ...
22
votes
2
answers
3k
views
Academic reference on the drawbacks of accuracy, F1 score, sensitivity and/or specificity
Accuracy, as a KPI for assessing binary classification models, has major drawbacks: Why is accuracy not the best measure for assessing classification models?. The exact same issues also plague the F1 ...
- 131k
1
vote
0
answers
15
views
Best way to sum up and visualize the relation between results of 2 tests measuring the same outcome with 3 results
I have a dataset of different individuals who underwent 2 test that measure the same outcome with 3 possible results
not reduced
mildly reduced
highly reduced
I know that the distribution of the ...
- 11
0
votes
1
answer
87
views
How can Mass Spectrometry have high sensitivity but average reproducibility?
I am reading up on mass spec from this link: https://www.ebi.ac.uk/training/online/courses/metabolomics-introduction/designing-a-metabolomics-study/comparison-of-nmr-and-ms/
They state:
The two most ...
- 367
1
vote
0
answers
48
views
Sensitivity and specificity for multiple raters
I am conducting a study where 3 raters assessed performance of two similar test in regards to classify a lesion as malignant vs. benign. I wish to calculate specificity and sensitivity of each test ...
- 11
0
votes
0
answers
72
views
How to apply Bayes' theorem on a contingency in which the sampled distribution differs drastically from the entire population?
Summary: How can I calculate my probability of being truly positive based on a contingency table that has a completely different distribution than the total population?
I was recently confronted with ...
- 1
0
votes
1
answer
416
views
Intersection over union vs sensitivity
In the context of segmentation, what is the difference between IoU and sensitivity? It sounds to me like they describe the same formula in different contexts but I might be wrong.
When true ...
- 101
0
votes
0
answers
95
views
Automatically obtain optimum cutoff point for accuracy, sensitivity and specificity
Below is a code snippet to plot the accuracy, sensitivity and specificity. I can manually eyeball and see the cutoff point at approximately around 0.3. I was wondering if there anyway to print out ...
5
votes
4
answers
5k
views
Precision vs. specificity
I know that if we cannot afford to have false positive results, we should aim for high precision.
My question is, how is precision different from specificity?
Any examples?
- 71
2
votes
2
answers
1k
views
Sensitivity vs. specificity vs. recall
Given a binary confusion matrix with true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN), what are the formulas for sensitivity, specificity, and recall?
I'm ...
- 219
1
vote
1
answer
63
views
What is the probability of at least one person in a sample of k people being infected?
Given the prevalence, sensitivity and specificity of a single test are known.
- 111
0
votes
0
answers
24
views
Sensitivity to settings vs statistical sensitivity
I am confused between two different properties of a model. Both are called sensitivity. First, sensitivity is another word for true positive rate. Second, the output of a model changes in response to ...
- 97
3
votes
1
answer
159
views
Is a Sensitivity-Specificity curve equal to a horizontally flipped ROC?
So I need to plot a Sensitivity-Specificity curve. Since ROCs represent TPR (sensitivity) against FPR (or 1 - Specificity), can I just plot TPR against 1 - FPR as in the code below to obtain a Sens-...
- 31
1
vote
1
answer
64
views
Why account for censoring in estimating sensitivity but not specificity?
I am reading Tutz & Schmid "Modeling Discrete Time-to-Event Data" (2016) chapter 4 Evaluation and Model Choice section 4.3.3 Discrimination Measures. On p. 94 estimators for sensitivity ...
- 69.5k
3
votes
2
answers
108
views
Calculating "accuracy", "recall" etc. without classification
I have a set of models, that I'm comparing to each other with respect to prediction of a binary event. I'm using a few proper scores (Brier, log), but I also need accuracy, recall, sensitivity etc., ...
- 1,127
2
votes
2
answers
49
views
Using a ROC Plot to interpret specific scores
I have a binary classifier which outputs a given score to differentiate normal (low score) from abnormal (high score) cases. The score itself however is non-interpretable to others.
I know a ROC plot ...
- 1,486
1
vote
2
answers
89
views
Individual Measurement in Overlapping Distributions
I have a situation where I have measurements taken from samples of healthy and pathological populations. The pathological populations tend to have much higher measurements, but the distributions are ...
- 1,486
1
vote
1
answer
34
views
How to show that simple random sample sensitivity is unbiased for population sensitivity
In diagnostic testing, sensitivity $S$ is the probability that the test gives a positive result given that you have the condition being tested. From a simple random sample of people who take the test, ...
0
votes
1
answer
180
views
Confidence intervals for series of Covid tests
Suppose I take two antigen tests and each test has a specificity of 75.3% (95% confidence interval from 65.8% to 83.4%) e.g. from https://pubmed.ncbi.nlm.nih.gov/33455451/
A. what is the sensitivity ...
- 1