# Questions tagged [sensitivity-specificity]

Sensitivity & specificity are descriptive statistics that measure the quality of a classification system. They are the proportion of real positives that are classified correctly (sensitivity) & the proportion of real negatives that are classified correctly (specificity). They are an integral part of ROC analysis.

219 questions
Filter by
Sorted by
Tagged with
7 views

### How to draw a ROC curve given estimated probability that a unit is positive and actual observed class? [duplicate]

Assume that a classification model fitted to data available to you has provided for each statistical unit a probability $P(+|x)$ that the unit is positive. The following table shows all available ...
12 views

### How can we apply sensitivity analysis in comparing two ARIMA models, both measuring intervention effect?

I have a dataset of reporting rates collected over an 8-month period obtained from the baseline (4 months) and during intervention (4 months). The goal is to determine if the intervention has a ...
6 views

### How to compute the minimum sensitivity and specificity a classifier needs to estimate a proportion to within some percent error?

I am building a binary classifier. Once classified, the proportion of positive events will be calculated. The predicted proportion is required to be within 10% of the true proportion. In practice, ...
• 1
1 vote
9 views

### Interpreting sobol indices

I am trying to test the sensitivity of my model to a change in the parameters std and translation_factor. I get the following results : Sobol First Order Indices: translation_factor: -1....
1 vote
99 views

### Issues with metaprop function for computing pooled specificity in meta-analysis

I am training to learn how to perform meta-analysis using R. I conducted a meta-analysis using the metaprop function in R with the provided dataset. The goal is to ...
75 views

### ROCAUC = average sensitivity across all thresholds according to IEEE TPAMI, yet my calculations show otherwise

Carrington et al (2023) make the claim that area under the receiver-operator characteristic curve is equal to the average sensitivity across all thresholds, and similarly for specificity (section 3), ...
• 64.2k
154 views

### Calculate area under precision-recall curve from area under ROC curve and the prevalence

I am reading material that reports the area under a ROC curve. I am curious to know what the performance would be in precision-recall space. From the sensitivity and specificity values in the ROC ...
• 64.2k
9 views

### Different sensitivity and specificity for balanced populations

I am building a simple test to flag prehistorical hand prints as belonging men or women. Using a random variable built from the fingers lengths, and having measured a population, I built the ROC curve ...
18 views

### Optimizing Threshold Selection for Improved Sensitivity in Classification Method Without Validated data

Assume we have a datasheet X, this datasheet contains many of samples with different gorup like G1, ...
• 113
64 views

### What should I do when a test specificity is zero?

Upon doing a test that compares the effect of imaging versus regular tissue biopsy for cancer that has a prevalence of 90%, we ...
39 views

### What is the relationship between bias-variance and sensitivity-specificity for novelty detection?

An over or under-parameterized binary classification model (- vs +) tends to over or under-fit (bias-variance tradeoff). This leads to errors during prediction on unseen data. Depending on if ...
1 vote
16 views

### Is there an equivalent for Yates' correction for a confusion matrix-derived metrics?

Given the following table of predictions vs. actual states: ...
• 1,099
93 views

### Binary classification metrics - Combining sensitivity and specificity?

The harmonic mean between precision and recall (F1 score) is a common metric to evaluate binary classification. It is useful because it strikes a balance between precision (FP) and recall (FN). For ...
• 1,247
104 views

### What is $\frac{a}{b}$ called in a $y=a x+b$ regression in the context of a physical detection test?

Let's say we have 10 calibrated reference samples of chemical products with a known concentration $x_i$ of a certain chemical component A. $x_i$ is different for each sample. We are building a ...
• 588
34 views

### How to do a sensitivity analysis for a Bayesian Model? Prior or Posterior?

I'm developing a mathematical (SIR-style) model for cholera transmission in the Bengal Delta which is fit to data using MCMC. The purpose of the model is more explanatory rather than predictive, and I ...
195 views

### Can Accuracy be higher than both sensitivity and specificity?

I came across a paper which reported the following results Accuracy Specificity Sensitivity 97.49% 93.6% 94.3% It seems unusual for accuracy to be higher than both sensitivity and specificity. Is ...
• 163
89 views

### How to obtain positive predictive values from multinomial regression?

For a 2x2 scenario, to determine misclassification we can obtain positive and negative predictive values from a logistic regression model where the outcome is the true value and the exposure is the ...
1 vote
131 views

### Why don't we use the harmonic mean of sensitivity and specificity?

There is this question on the F-1 score, asking why we compute the harmonic mean of precision and recall rather than its arithmetic mean. There were good arguments in the answers in favor of the ...
• 211
151 views

### Comparing Sensitivity, Specificity, PPV and NPV of one test across two subgroups

I am seeking to compare the diagnostic performance of a test within a population with two defined subgroups. I have the raw data & computed data, but am unsure what the best test is to compare the ...
1k views

### pROC package - sensitivity and specificity calculations

I am using the pROC package in R to generate ROC curves. Using the "coords" function, I can extract the sensitivity (Se) , specificity (Sp), negative predicted value (NPV) and positive ...
• 149
72 views

### AUC - different interpretations

When browsing through literature about ROC - AUC, there seems to be a disparity. While some plot TPR and FPR, e.g. from Wikipedia: "The ROC curve is created by plotting the true positive rate (...
349 views

### How to select a model based on ROC AUC, sensitivity and specificity?

I'm running several machine learning algorithms on a dataset with 80% negatives and 20% positive cases (classification). Below I attach the results of comparing performance on 500 bootstrap resamples ...
• 13
1 vote
570 views

### (Why) Does positive predictive value depend on sensitivity?

Assume there are two possible categories, coded as $0$ and $1$. Define positive predictive value (PPV) to be the probability that an observation really is in category $1$ when it is predicted as being ...
• 64.2k
408 views

### Calculating the Brier or log score from the confusion matrix, or from accuracy, sensitivity, specificity, F1 score etc

Suppose I have a confusion matrix, or alternatively any one or more of accuracy, sensitivity, specificity, recall, F1 score or friends for a binary classification problem. How can I calculate the ...
• 126k
1 vote
318 views

### Comparing diagnostic test in different populations

I have 5 diagnostic tests (test A, B, C and D) for a certain disease, tested in 2 different populations. For comparing test parameters (sensitivity, specificity) of test A and B in the same population ...
2k views

### Academic reference on the drawbacks of accuracy, F1 score, sensitivity and/or specificity

Accuracy, as a KPI for assessing binary classification models, has major drawbacks: Why is accuracy not the best measure for assessing classification models?. The exact same issues also plague the F1 ...
• 126k
1 vote
14 views

### Best way to sum up and visualize the relation between results of 2 tests measuring the same outcome with 3 results

I have a dataset of different individuals who underwent 2 test that measure the same outcome with 3 possible results not reduced mildly reduced highly reduced I know that the distribution of the ...
• 11
74 views

### How can Mass Spectrometry have high sensitivity but average reproducibility?

I am reading up on mass spec from this link: https://www.ebi.ac.uk/training/online/courses/metabolomics-introduction/designing-a-metabolomics-study/comparison-of-nmr-and-ms/ They state: The two most ...
• 367
1 vote
38 views

### Sensitivity and specificity for multiple raters

I am conducting a study where 3 raters assessed performance of two similar test in regards to classify a lesion as malignant vs. benign. I wish to calculate specificity and sensitivity of each test ...
• 11
56 views

### How to apply Bayes' theorem on a contingency in which the sampled distribution differs drastically from the entire population?

Summary: How can I calculate my probability of being truly positive based on a contingency table that has a completely different distribution than the total population? I was recently confronted with ...
376 views

### Intersection over union vs sensitivity

In the context of segmentation, what is the difference between IoU and sensitivity? It sounds to me like they describe the same formula in different contexts but I might be wrong. When true ...
• 101
76 views

### Automatically obtain optimum cutoff point for accuracy, sensitivity and specificity

Below is a code snippet to plot the accuracy, sensitivity and specificity. I can manually eyeball and see the cutoff point at approximately around 0.3. I was wondering if there anyway to print out ...
3k views

### Precision vs. specificity

I know that if we cannot afford to have false positive results, we should aim for high precision. My question is, how is precision different from specificity? Any examples?
• 71
776 views

### Sensitivity vs. specificity vs. recall

Given a binary confusion matrix with true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN), what are the formulas for sensitivity, specificity, and recall? I'm ...
• 219
1 vote
58 views

### What is the probability of at least one person in a sample of k people being infected?

Given the prevalence, sensitivity and specificity of a single test are known.
• 111
23 views

### Sensitivity to settings vs statistical sensitivity

I am confused between two different properties of a model. Both are called sensitivity. First, sensitivity is another word for true positive rate. Second, the output of a model changes in response to ...
• 97
137 views

### Is a Sensitivity-Specificity curve equal to a horizontally flipped ROC?

So I need to plot a Sensitivity-Specificity curve. Since ROCs represent TPR (sensitivity) against FPR (or 1 - Specificity), can I just plot TPR against 1 - FPR as in the code below to obtain a Sens-...
1 vote
55 views

### Why account for censoring in estimating sensitivity but not specificity?

I am reading Tutz & Schmid "Modeling Discrete Time-to-Event Data" (2016) chapter 4 Evaluation and Model Choice section 4.3.3 Discrimination Measures. On p. 94 estimators for sensitivity ...
• 68.3k
98 views

### Calculating "accuracy", "recall" etc. without classification

I have a set of models, that I'm comparing to each other with respect to prediction of a binary event. I'm using a few proper scores (Brier, log), but I also need accuracy, recall, sensitivity etc., ...
48 views

### Using a ROC Plot to interpret specific scores

I have a binary classifier which outputs a given score to differentiate normal (low score) from abnormal (high score) cases. The score itself however is non-interpretable to others. I know a ROC plot ...
• 1,486
1 vote
69 views

### Individual Measurement in Overlapping Distributions

I have a situation where I have measurements taken from samples of healthy and pathological populations. The pathological populations tend to have much higher measurements, but the distributions are ...
• 1,486
1 vote
33 views

### How to show that simple random sample sensitivity is unbiased for population sensitivity

In diagnostic testing, sensitivity $S$ is the probability that the test gives a positive result given that you have the condition being tested. From a simple random sample of people who take the test, ...
176 views

### Confidence intervals for series of Covid tests

Suppose I take two antigen tests and each test has a specificity of 75.3% (95% confidence interval from 65.8% to 83.4%) e.g. from https://pubmed.ncbi.nlm.nih.gov/33455451/ A. what is the sensitivity ...
39 views

### Post test probability with a ROC curve

I have data that is normally distributed related to risk of a particular disease. At the median of the distribution, you would expect to observe the population prevalence level of disease P0=0.01. For ...
243 views

### Calculating true positive and true negative from sensitivity and specificity

I know the specificity and sensitivity of a test are both 90%. The incidence is 2.5%. The total population is 10,000 and the number of people with the illness is 250, how do I calculate the true ...
857 views

### Logistic regression metric

I am interested to understand in which scenarios person should use sensitivity, specificity, and when should person opt for precision recall. On a high level I understand for a balanced data set we ...
1 vote
88 views

### Is Average Accuracy An Ill-Defined Metric? [closed]

Five people each took a 100 question yes/no question test. I have their individual accuracy scores and was planning to report their average. However, I was informed that "You cannot take an ...
• 33
611 views

### Create AUC-ROC from single sensitivity and specificity value? [duplicate]

Is it possible and appropriate to estimate the area under the receiver operating characteristic curve from a single point estimate of an individual's sensitivity and specificity performance?
• 33