Skip to main content

Questions tagged [sensitivity-specificity]

Sensitivity & specificity are descriptive statistics that measure the quality of a classification system. They are the proportion of real positives that are classified correctly (sensitivity) & the proportion of real negatives that are classified correctly (specificity). They are an integral part of ROC analysis.

Filter by
Sorted by
Tagged with
0 votes
0 answers
7 views

How to draw a ROC curve given estimated probability that a unit is positive and actual observed class? [duplicate]

Assume that a classification model fitted to data available to you has provided for each statistical unit a probability $P(+|x)$ that the unit is positive. The following table shows all available ...
evans5's user avatar
  • 3
0 votes
0 answers
12 views

How can we apply sensitivity analysis in comparing two ARIMA models, both measuring intervention effect?

I have a dataset of reporting rates collected over an 8-month period obtained from the baseline (4 months) and during intervention (4 months). The goal is to determine if the intervention has a ...
George Mwenye-Phiri's user avatar
0 votes
0 answers
6 views

How to compute the minimum sensitivity and specificity a classifier needs to estimate a proportion to within some percent error?

I am building a binary classifier. Once classified, the proportion of positive events will be calculated. The predicted proportion is required to be within 10% of the true proportion. In practice, ...
Dave's user avatar
  • 1
1 vote
0 answers
9 views

Interpreting sobol indices

I am trying to test the sensitivity of my model to a change in the parameters std and translation_factor. I get the following results : Sobol First Order Indices: translation_factor: -1....
Anem Dupré's user avatar
1 vote
1 answer
99 views

Issues with metaprop function for computing pooled specificity in meta-analysis

I am training to learn how to perform meta-analysis using R. I conducted a meta-analysis using the metaprop function in R with the provided dataset. The goal is to ...
Reza Khayami's user avatar
6 votes
1 answer
75 views

ROCAUC = average sensitivity across all thresholds according to IEEE TPAMI, yet my calculations show otherwise

Carrington et al (2023) make the claim that area under the receiver-operator characteristic curve is equal to the average sensitivity across all thresholds, and similarly for specificity (section 3), ...
Dave's user avatar
  • 64.2k
3 votes
2 answers
154 views

Calculate area under precision-recall curve from area under ROC curve and the prevalence

I am reading material that reports the area under a ROC curve. I am curious to know what the performance would be in precision-recall space. From the sensitivity and specificity values in the ROC ...
Dave's user avatar
  • 64.2k
0 votes
0 answers
9 views

Different sensitivity and specificity for balanced populations

I am building a simple test to flag prehistorical hand prints as belonging men or women. Using a random variable built from the fingers lengths, and having measured a population, I built the ROC curve ...
Xavier Prudent's user avatar
0 votes
0 answers
18 views

Optimizing Threshold Selection for Improved Sensitivity in Classification Method Without Validated data

Assume we have a datasheet X, this datasheet contains many of samples with different gorup like G1, ...
zhang's user avatar
  • 113
0 votes
0 answers
64 views

What should I do when a test specificity is zero?

Upon doing a test that compares the effect of imaging versus regular tissue biopsy for cancer that has a prevalence of 90%, we ...
Mohamed Rahouma's user avatar
0 votes
0 answers
39 views

What is the relationship between bias-variance and sensitivity-specificity for novelty detection?

An over or under-parameterized binary classification model (- vs +) tends to over or under-fit (bias-variance tradeoff). This leads to errors during prediction on unseen data. Depending on if ...
Douw Marx's user avatar
1 vote
1 answer
16 views

Is there an equivalent for Yates' correction for a confusion matrix-derived metrics?

Given the following table of predictions vs. actual states: ...
Bryan's user avatar
  • 1,099
2 votes
1 answer
93 views

Binary classification metrics - Combining sensitivity and specificity?

The harmonic mean between precision and recall (F1 score) is a common metric to evaluate binary classification. It is useful because it strikes a balance between precision (FP) and recall (FN). For ...
usual me's user avatar
  • 1,247
4 votes
1 answer
104 views

What is $\frac{a}{b}$ called in a $y=a x+b$ regression in the context of a physical detection test?

Let's say we have 10 calibrated reference samples of chemical products with a known concentration $x_i$ of a certain chemical component A. $x_i$ is different for each sample. We are building a ...
Basj's user avatar
  • 588
3 votes
0 answers
34 views

How to do a sensitivity analysis for a Bayesian Model? Prior or Posterior?

I'm developing a mathematical (SIR-style) model for cholera transmission in the Bengal Delta which is fit to data using MCMC. The purpose of the model is more explanatory rather than predictive, and I ...
Learning_how_to_model's user avatar
5 votes
1 answer
195 views

Can Accuracy be higher than both sensitivity and specificity?

I came across a paper which reported the following results Accuracy Specificity Sensitivity 97.49% 93.6% 94.3% It seems unusual for accuracy to be higher than both sensitivity and specificity. Is ...
Jack O'Neill's user avatar
2 votes
2 answers
89 views

How to obtain positive predictive values from multinomial regression?

For a 2x2 scenario, to determine misclassification we can obtain positive and negative predictive values from a logistic regression model where the outcome is the true value and the exposure is the ...
lauraellen's user avatar
1 vote
2 answers
131 views

Why don't we use the harmonic mean of sensitivity and specificity?

There is this question on the F-1 score, asking why we compute the harmonic mean of precision and recall rather than its arithmetic mean. There were good arguments in the answers in favor of the ...
user209974's user avatar
0 votes
0 answers
151 views

Comparing Sensitivity, Specificity, PPV and NPV of one test across two subgroups

I am seeking to compare the diagnostic performance of a test within a population with two defined subgroups. I have the raw data & computed data, but am unsure what the best test is to compare the ...
Richard's user avatar
3 votes
1 answer
1k views

pROC package - sensitivity and specificity calculations

I am using the pROC package in R to generate ROC curves. Using the "coords" function, I can extract the sensitivity (Se) , specificity (Sp), negative predicted value (NPV) and positive ...
Falco's user avatar
  • 149
2 votes
1 answer
72 views

AUC - different interpretations

When browsing through literature about ROC - AUC, there seems to be a disparity. While some plot TPR and FPR, e.g. from Wikipedia: "The ROC curve is created by plotting the true positive rate (...
Stani Petrov's user avatar
0 votes
1 answer
349 views

How to select a model based on ROC AUC, sensitivity and specificity?

I'm running several machine learning algorithms on a dataset with 80% negatives and 20% positive cases (classification). Below I attach the results of comparing performance on 500 bootstrap resamples ...
amr95's user avatar
  • 13
1 vote
2 answers
570 views

(Why) Does positive predictive value depend on sensitivity?

Assume there are two possible categories, coded as $0$ and $1$. Define positive predictive value (PPV) to be the probability that an observation really is in category $1$ when it is predicted as being ...
Dave's user avatar
  • 64.2k
8 votes
2 answers
408 views

Calculating the Brier or log score from the confusion matrix, or from accuracy, sensitivity, specificity, F1 score etc

Suppose I have a confusion matrix, or alternatively any one or more of accuracy, sensitivity, specificity, recall, F1 score or friends for a binary classification problem. How can I calculate the ...
Stephan Kolassa's user avatar
1 vote
1 answer
318 views

Comparing diagnostic test in different populations

I have 5 diagnostic tests (test A, B, C and D) for a certain disease, tested in 2 different populations. For comparing test parameters (sensitivity, specificity) of test A and B in the same population ...
Adrián Valls Carbó's user avatar
21 votes
2 answers
2k views

Academic reference on the drawbacks of accuracy, F1 score, sensitivity and/or specificity

Accuracy, as a KPI for assessing binary classification models, has major drawbacks: Why is accuracy not the best measure for assessing classification models?. The exact same issues also plague the F1 ...
Stephan Kolassa's user avatar
1 vote
0 answers
14 views

Best way to sum up and visualize the relation between results of 2 tests measuring the same outcome with 3 results

I have a dataset of different individuals who underwent 2 test that measure the same outcome with 3 possible results not reduced mildly reduced highly reduced I know that the distribution of the ...
Guddi's user avatar
  • 11
0 votes
1 answer
74 views

How can Mass Spectrometry have high sensitivity but average reproducibility?

I am reading up on mass spec from this link: https://www.ebi.ac.uk/training/online/courses/metabolomics-introduction/designing-a-metabolomics-study/comparison-of-nmr-and-ms/ They state: The two most ...
statHacker's user avatar
1 vote
0 answers
38 views

Sensitivity and specificity for multiple raters

I am conducting a study where 3 raters assessed performance of two similar test in regards to classify a lesion as malignant vs. benign. I wish to calculate specificity and sensitivity of each test ...
PalpatR's user avatar
  • 11
0 votes
0 answers
56 views

How to apply Bayes' theorem on a contingency in which the sampled distribution differs drastically from the entire population?

Summary: How can I calculate my probability of being truly positive based on a contingency table that has a completely different distribution than the total population? I was recently confronted with ...
jotheta's user avatar
0 votes
1 answer
376 views

Intersection over union vs sensitivity

In the context of segmentation, what is the difference between IoU and sensitivity? It sounds to me like they describe the same formula in different contexts but I might be wrong. When true ...
Yiftach's user avatar
  • 101
0 votes
0 answers
76 views

Automatically obtain optimum cutoff point for accuracy, sensitivity and specificity

Below is a code snippet to plot the accuracy, sensitivity and specificity. I can manually eyeball and see the cutoff point at approximately around 0.3. I was wondering if there anyway to print out ...
Larry Chuon's user avatar
5 votes
4 answers
3k views

Precision vs. specificity

I know that if we cannot afford to have false positive results, we should aim for high precision. My question is, how is precision different from specificity? Any examples?
MxML's user avatar
  • 71
2 votes
2 answers
776 views

Sensitivity vs. specificity vs. recall

Given a binary confusion matrix with true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN), what are the formulas for sensitivity, specificity, and recall? I'm ...
user72716's user avatar
  • 219
1 vote
1 answer
58 views

What is the probability of at least one person in a sample of k people being infected?

Given the prevalence, sensitivity and specificity of a single test are known.
r3dapple's user avatar
  • 111
0 votes
0 answers
23 views

Sensitivity to settings vs statistical sensitivity

I am confused between two different properties of a model. Both are called sensitivity. First, sensitivity is another word for true positive rate. Second, the output of a model changes in response to ...
Mikhail's user avatar
  • 97
3 votes
1 answer
137 views

Is a Sensitivity-Specificity curve equal to a horizontally flipped ROC?

So I need to plot a Sensitivity-Specificity curve. Since ROCs represent TPR (sensitivity) against FPR (or 1 - Specificity), can I just plot TPR against 1 - FPR as in the code below to obtain a Sens-...
evenigrammer's user avatar
1 vote
1 answer
55 views

Why account for censoring in estimating sensitivity but not specificity?

I am reading Tutz & Schmid "Modeling Discrete Time-to-Event Data" (2016) chapter 4 Evaluation and Model Choice section 4.3.3 Discrimination Measures. On p. 94 estimators for sensitivity ...
Richard Hardy's user avatar
3 votes
2 answers
98 views

Calculating "accuracy", "recall" etc. without classification

I have a set of models, that I'm comparing to each other with respect to prediction of a binary event. I'm using a few proper scores (Brier, log), but I also need accuracy, recall, sensitivity etc., ...
Accidental Statistician's user avatar
2 votes
2 answers
48 views

Using a ROC Plot to interpret specific scores

I have a binary classifier which outputs a given score to differentiate normal (low score) from abnormal (high score) cases. The score itself however is non-interpretable to others. I know a ROC plot ...
DankMasterDan's user avatar
1 vote
2 answers
69 views

Individual Measurement in Overlapping Distributions

I have a situation where I have measurements taken from samples of healthy and pathological populations. The pathological populations tend to have much higher measurements, but the distributions are ...
DankMasterDan's user avatar
1 vote
1 answer
33 views

How to show that simple random sample sensitivity is unbiased for population sensitivity

In diagnostic testing, sensitivity $S$ is the probability that the test gives a positive result given that you have the condition being tested. From a simple random sample of people who take the test, ...
variancekills's user avatar
0 votes
1 answer
176 views

Confidence intervals for series of Covid tests

Suppose I take two antigen tests and each test has a specificity of 75.3% (95% confidence interval from 65.8% to 83.4%) e.g. from https://pubmed.ncbi.nlm.nih.gov/33455451/ A. what is the sensitivity ...
sdflack's user avatar
0 votes
0 answers
39 views

Post test probability with a ROC curve

I have data that is normally distributed related to risk of a particular disease. At the median of the distribution, you would expect to observe the population prevalence level of disease P0=0.01. For ...
sethsh7's user avatar
0 votes
0 answers
243 views

Calculating true positive and true negative from sensitivity and specificity

I know the specificity and sensitivity of a test are both 90%. The incidence is 2.5%. The total population is 10,000 and the number of people with the illness is 250, how do I calculate the true ...
AliceH's user avatar
  • 1
2 votes
2 answers
857 views

Logistic regression metric

I am interested to understand in which scenarios person should use sensitivity, specificity, and when should person opt for precision recall. On a high level I understand for a balanced data set we ...
Teja Bandaru's user avatar
1 vote
1 answer
88 views

Is Average Accuracy An Ill-Defined Metric? [closed]

Five people each took a 100 question yes/no question test. I have their individual accuracy scores and was planning to report their average. However, I was informed that "You cannot take an ...
pw-314's user avatar
  • 33
2 votes
1 answer
611 views

Create AUC-ROC from single sensitivity and specificity value? [duplicate]

Is it possible and appropriate to estimate the area under the receiver operating characteristic curve from a single point estimate of an individual's sensitivity and specificity performance?
pw-314's user avatar
  • 33
0 votes
0 answers
20 views

Roc curve straight line interpretation [duplicate]

How can I fix the straight line? is that mean that 70% of my cases that tag as positive and actually negative have the same value?
user348733's user avatar
1 vote
1 answer
365 views

Accuracy, Recall (sensitivity), Specificity confusion

I came up with this paper with 765 citations! On page 2, it expresses an equation relating these metrics: Accuracy sensitivity (a.k.a Recall). specificity (a.k.a 1 - type1 error rate) prevalence (...
Ehsan Sh's user avatar
  • 525

1
2 3 4 5