Questions tagged [sensitivity-specificity]

Sensitivity & specificity are descriptive statistics that measure the quality of a classification system. They are the proportion of real positives that are classified correctly (sensitivity) & the proportion of real negatives that are classified correctly (specificity). They are an integral part of ROC analysis.

51 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3
votes
1answer
69 views

Why we use precision/recall in binary classification but sensitivity(=recall)/specificity in medicine?

Sensitivity=recall is used in both fields, but the second metric is different. Why? Both tasks (classification and medicine) look same - data has two classes and we do some predictions on it and want ...
3
votes
0answers
81 views

What is the correct definition of Selectivity : TPR or TNR ? Since contradictory literature exists

The reason of my question is because contradictory articles exist. Talking about the confusion matrix's performance metrics, we can find: English wikipedia pages mention that ...
3
votes
0answers
163 views

Assessing correlated predictions

Let's assume we have a prediction algorithm (if it helps, imagine it's using some boosted tree method) that does daily predictions for whether some event will happen to a unit (e.g. a machine that ...
3
votes
1answer
271 views

The effect of oversampling on the positive predictive value

I need to calculate the positive predictive value for a validation set for a rare event. The problem is that the validation set was oversampled for the rare event. The event occurs in 5 percent of the ...
2
votes
0answers
34 views

Origin of terms “sensitivity” and “specificity”

Who coined the terms "sensitivity" and "specificity"—meaning the complements of false positives, and false negatives, respectively in tests and measures—and when did they first do ...
2
votes
0answers
25 views

Prevalence threshold calculations error

I see on this page the formula for calculation of prevalence threshold: Given also are following values on that page: ...
2
votes
1answer
142 views

ROC Curve AUC for Hypothesis Testing Sensitivity (Power) vs Specificity ($1-\alpha$)

Let's set up a hypothesis test of $H_0: \theta=\theta_0$ versus $H_1: \theta\ne\theta_0$, and let's say that I have two techniques to assess this (say equal-variance t-test versus unequal-variance ...
2
votes
0answers
981 views

what to do when true positive rate is undefined?

I have a binary classification set with about 900 samples. I first use kfold cross validation and AUC of the ROC curve to determine which classifiers perform best. Once I've done that, I use a leave-...
2
votes
0answers
126 views

Calculating and reporting accuracy, sensitivity and specificity with multiple raters

I'm conducting a study in which I'm comparing 15 radiologists evaluations on a dichotomous variable to a gold standard, in a sample with 58 patients. Obviously I've calculated overall accuracy, ...
2
votes
1answer
35 views

Is this a sensitivity?

There are test A and test B. I want to show a ratio of test A positive among all test positive(test A positive or test B positive). Can I call the ratio as sensitivity? Or is it safe to call as ratio?...
2
votes
0answers
1k views

How to improve the sensitivity of minority class on imbalanced datasets

I am working on a classifier which stratifies a population of samples into different classes. The class distribution (ground truth) is imbalanced, and the prevalence of each class is: $$\begin{...
2
votes
0answers
59 views

Diagnostic device hypothesis testing for superiority to a fixed value

I am designing a study to demonstrate that a diagnostic medical device is more sensitive than a fixed performance goal of 70%. Currently I am attempting to determine the appropriate sample size. My ...
1
vote
0answers
7 views

Tuning the Logit model based on sensitivity

I am working on a logistic model to predict whether or not an employee is likely to take a leave day during a given three month period. The model only correctly classified the “non-event" ...
1
vote
0answers
15 views

Take into account Sensitivity and Specificity when calculating Estimates/CI's

I'm trying to thing about how to calculate confidence intervals in the presence of known measurement error for some test protocol. Let's assume we're doing a sample survey in which we're performing ...
1
vote
0answers
26 views

Population selection for sensitivity and specificity

For medical physical assessment sensitivity and specificity is commonly used to assess the clinical relevance of the test. I'm specifically referring to manual medicine here, with orthopedic tests of ...
1
vote
1answer
93 views

Bootstrap vs Wilson score confidence interval

For estimation of the confidence interval of sensitivity and specificity, when I should use the Wilson score and when I should use bootstrapping?
1
vote
0answers
17 views

OneR and area under the ROC curve

In the book Data Mining, author Ian Witten says that it does not make sense to calculate the area under the ROC curve when applying OneR because it finds only one point (sensitivity, specificity) ...
1
vote
0answers
14 views

Sensitivity of a diagnostic test

Diagnostic tests A1 and A2 are used to detect the presence or absence of a disease. The results of A1 and A2 are a priori independent of the presence or absence of the disease. A1 and A2 have ...
1
vote
0answers
36 views

Deriving a loss function properly accounting for different error type costs

Consider a classification setting like a medical test: Not finding an existing health issue might be much worse (by a factor of 50) than assuming an issue when there is none. I.e. a setting in which ...
1
vote
0answers
20 views

Sensitivity and specificity on cases rather than individual days

I have data collected from sensors to detect disease in several individuals. Each individual has a known disease status (labelled Actual in the table below) for ...
1
vote
0answers
26 views

Compare cut-offs with respect to sensitivity etc

I have a score and a dichotomous criterion for n observations. Using a ROC-Analysis, I found some optimal cut-off, say c. Now, literature (e.g., author of scale with which the score was determined) ...
1
vote
0answers
535 views

Calculating sensitivity and specificity from survival data

I have a diagnostic test performed on 100 participants at baseline. I then follow up these participants for variable periods of time and have data regarding survival. I have used a Cox regression ...
1
vote
1answer
331 views

Sensitivity/Specificity compared to Area Under Curve as measure of Screen Accuracy

In relation to the disorder I'm studying Screen A is reported as having a sensitivity of 90% and a specificity of 89%. Screen B is reported as having a AUC of .79 with no other data provided. Could ...
1
vote
0answers
75 views

How to make specificity a stable evaluation metric?

ROC (including AUC) metrics are widely used for binary classification problems. AUC is usually selected to evaluate the model. However, some tasks may require high specificity with fixed sensitivity. ...
1
vote
0answers
549 views

Calculating a pooled estimate

I am writing a systematic review on certain imaging studies and I was hoping to add a meta-analysis of the sensitivities of the various modalities reported on. My goal was to put together a forest ...
1
vote
0answers
70 views

Confidence interpretation from false negative rate

For my research I'm using a tool called seeSUMO, which predicts sumoylation sites in a protein based on sequence features. When it reports your results, it gives you a level of confidence for each ...
0
votes
0answers
15 views

Question on Machine Learning to Generate Genetic Algorithms with Highest Sensitivity and Specificity Values

I am new to this website, but I am a researcher who has no experience in ML who is trying to generate molecular algorithms that use a combination of genetic markers in order to correctly predict ...
0
votes
0answers
9 views

Statistical metric for maximizing true positives relative to “redundant” positives

I am trying to optimize a binary classification test. Unlike many common instances, my "list" of true positives and false positives also contains a large number of redundant true positives. ...
0
votes
0answers
11 views

What could be a good way to interpret this neurophysiological data?

We are looking at different combinations of neurophysiological tests, and trying to find out which combinations of variables are best suited for clinical use (diagnosing diabetic polyneuropathy). To ...
0
votes
1answer
89 views

Identifying threshold from Youden Index - Using ROC curve to calculate minimally important change (MIC)

I am trying to determine the minimally important change (MIC) of a frailty instrument using an anchor-based approach outlined below. Step 1. Fit a logistic regression model between ...
0
votes
0answers
9 views

compute posterior probability using specificity and sensitivity

Im trying to calculate the posterior and have this posterior_probability = (prevalence * sensitivity)/ ((prevalence * sensitivity) + ((1-prevalence) * specificity)) but am getting the wrong answer?
0
votes
0answers
23 views

Can AUC increase while both sensitivity and specificity decrease?

Assume I am given two classifiers - A and B. Is it possible that the specificity and sensitivity obtained from using B will be higher than A, but the AUC of A will be higher than the AUC of B? Will ...
0
votes
0answers
9 views

Variable significance very sensitive to specification of non-correlated second variable

I´m doing research on a political science topic and my models leave me behind with a big questionmark at this point. I have a dataset containing 79 observations on a number of variables and trying and ...
0
votes
0answers
40 views

P-values for sensitivity/specificity when only improvement is possible

Imagine two diagnostic tests in a single population at risk of the disease, one of which is always "more strict" than the other, meaning that if test 1 is positive, test 2 will be positive ...
0
votes
0answers
12 views

Testing the difference in sensitivities between two lab tests - with part overlapping populations

I have two rapid diagnostic tests for which I test their sensitivity (+/-) in detecting cases against the gold standard (ELISA, also +/-). I now want to test whether the tests perform significantly ...
0
votes
0answers
65 views

Ordinal logistic regression: sensitivity / specifity

I am trying to determine the sensitivity and specificity per category of an outcome with ordinal categories between 1 and 6 predicted by 5 predictors. Overall accuracy is around 45%, but I want to ...
0
votes
0answers
7 views

Sensitivity test

I have an independent variable that has 3 groups - 'Yes' 'No' and 'Unknown'. I have decided that I can assume the 'Unknowns' as 'No'. I want to run sensitivity analysis to test whether making this ...
0
votes
0answers
12 views

Can the smallest detectable change or minimal detectable change of a measure be applied to a study with a much smaller sample size?

In a pilot study, we are evaluating whether there was an effect of the intervention. To determine the magnitude of the effect we are comparing the mean difference between timepoints to the group level ...
0
votes
0answers
14 views

Imperfect test sensitivity…what does 0% prevalence really mean?

I'm trying to think about test sensitivity and specificity in a theoretical disease-host system. Say you have a population from which you take a sample and test for the pathogen of interest using a ...
0
votes
0answers
9 views

Could two tests, one with high sensitivity, and other with high specificity, be meaningfully combined?

As an example, let's say we have some test with 99% sensitivity but 50% specificity, and we have some other test with 60% sensitivity and 95% specificity. So, we could have confidence in positive ...
0
votes
0answers
313 views

Very low specificity in random forests classification model with balanced classes

I've been building a model within a k-fold cross-validation,and seem to be getting pretty much the same results i.e. very high sensitivity and low specificity. What I've tried: Different values of ...
0
votes
1answer
18 views

Combining time series to increase sensitivity

I have two sensors logging data synchronously. The two sensors are similar, so the recorded time series are correlated except for some special conditions. I recorded data in different input parameter ...
0
votes
0answers
44 views

Finding PCA-like directions in feature space that maximise sensitivity to a target variable

I have a fairly large space of feature variables in which I want to build a predictor for a target variable. My input dataset for training the predictor are sampled from the space using a mix of log ...
0
votes
0answers
28 views

Assess sensitivity of a binary screening result across other continuous covariate(s)

I want to assess the sensitivity of a diagnostic screening test, which is a binary test (screen +ve and screen -ve). There is no continuous measure from this test - it is literally yes or no. My ...
0
votes
0answers
32 views

estimate probability that sensitivity/specificity greater than threshold value

Is there a Bayesian method to estimate the probability that the sensitivity of a diagnostic test is greater than some value, say 0.70? Rather than estimate sensitivity and a confidence interval (e.g., ...
0
votes
0answers
64 views

Why do dummy regressors decrease sensitivity in linear regression?

For linear regression analysis, I thought that the addition of covariates which are not related to the dependent variable $Y$ does not decrease sensitivity. Such random/dummy regressors can be ...
0
votes
0answers
318 views

Applying SMOTE and increasing sensitivity

I am trying to analyze lending club data and want to predict whether a loan is risky or safe using random forest with decision tree as a classifier. The data is imbalanced. It contains one-fourth of ...
0
votes
0answers
50 views

What is the correct method for comparing sensitivity and specifity of different tests?

What is the correct method of comparing efficiency of different test for one sample of individuals? Are ROC and AUC enough? Comparing of sensitivity and specifity values of the tests with McNemar ...
0
votes
0answers
46 views

How to evaluate sensitivity and specifity of different tests

I am comparing two test using different cut-off values with golden standart test . How is it to be reported. Is it sufficent to report sensitivity, specivity, PPV and NPV with 95 CI or should I ...
0
votes
0answers
518 views

Cost sensitive analysis versus Threshold

I am running a cost sensitive ensemble methods (for the first time to access algorithms of this type) 'CostSensitiveRandomPatchesClassifier' in Python on my imbalanced dataset (positive cases are rare ...