Questions tagged [sensitivity-specificity]

Sensitivity & specificity are descriptive statistics that measure the quality of a classification system. They are the proportion of real positives that are classified correctly (sensitivity) & the proportion of real negatives that are classified correctly (specificity). They are an integral part of ROC analysis.

Filter by
Sorted by
Tagged with
0
votes
2answers
53 views

Calculate AUC using sensitivity and specificity values only

How to calculate AUC, if I have values of sensitivity and specificity for various threshold cutoffs? I have sensitivity and specificity values for 100 thresholds. ...
0
votes
0answers
7 views

Samle size calculation for paired comparative diagnostic accuracy study

I would like to estimate an appropriate sample size for a medical imaging study. I prefer to use R. A standard thest testS will be compared to a modified standard ...
0
votes
0answers
15 views

Finding PCA-like directions in feature space that maximise sensitivity to a target variable

I have a fairly large space of feature variables in which I want to build a predictor for a target variable. My input dataset for training the predictor are sampled from the space using a mix of log ...
1
vote
2answers
34 views

Sensitivity rates where TP =0 and FN =0

In calculating sensitivity rates where TP =0 and FN =0 and the formula is TP/(TP+FN) - although it mathematically won't compute, does this equate to 100% sensitivity since it has correctly identified ...
0
votes
1answer
22 views

Correct calculation of repeated cross-validation classification metrics

We can obtain a resampled estimate of training set classification accuracy from caret::confusionMatrix.train(model) e.g., ...
0
votes
0answers
8 views

Can positive and negative likelihood ratios be combined into one parameter?

Positive and negative likelihood ratios (PLR, NLR) are considered to be much better than sensitivity and specificity as parameters of usefulness of a test. Is it possible to combine PLR and NLR into ...
0
votes
0answers
17 views

Acceptable Accuracy, Precision, Sensitivity, and Specificity Thresholds [duplicate]

Are there general rule of thumbs for acceptable accuracy, precision, sensitivity, and specificity values/thresholds in classification? I would imagine that this depends on different applications. I ...
0
votes
1answer
21 views

Why are likelihood ratios weighted for prevalence?

I notice LLR's are weighted for prevalence if the prevalence differs from 0.5 (see: http://vassarstats.net/clin2.html) Can someone explain why this is? I'm a medical student, not a statistician, so I'...
0
votes
0answers
6 views

Can I estimate a new post-test probability?

Let's say a patient, let's assume a pre-test probability of 20%, gets a test done with a positive predictive value of 70%. He tests positive. He gets another test done, which has a positive predictive ...
0
votes
1answer
38 views

Is this test muddying the water?

In the case of a neurological disease called idiopathic normal pressure hydrocephalus, if the patient is suspected positive for the condition, they are administered a test called the spinal tap test. ...
3
votes
0answers
48 views

What is the correct definition of Selectivity : TPR or TNR ? Since contradictory literature exists

The reason of my question is because contradictory articles exist. Talking about the confusion matrix's performance metrics, we can find: English wikipedia pages mention that ...
0
votes
1answer
24 views

Plot of Probability of Cancer vs Test Sensitivity

I'm looking at the following problem and its solution and am really struggling to understand how they came up with the necessary values. My Questions 1, For part (a) we are asked to plot $y=P(C|+)$...
0
votes
0answers
18 views

Compute Sensitivity and Specificity with Incomplete Data

I have two data sets $A$ and $B$ which I want to combine using statistical methods (e.g. some matching algorithms such as Nearest Neighbor). I have a third data set $C$ that encompasses the true ...
0
votes
0answers
25 views

Assess sensitivity of a binary screening result across other continuous covariate(s)

I want to assess the sensitivity of a diagnostic screening test, which is a binary test (screen +ve and screen -ve). There is no continuous measure from this test - it is literally yes or no. My ...
0
votes
0answers
41 views

GLMM with rare events causing issues with specificity?

I have run a generalized linear mixed model (link=logit) and my outcome of interest is present in about 8% of observations. The total sample is 1,717,323 and of these 131,389 observations are where ...
0
votes
0answers
30 views

estimate probability that sensitivity/specificity greater than threshold value

Is there a Bayesian method to estimate the probability that the sensitivity of a diagnostic test is greater than some value, say 0.70? Rather than estimate sensitivity and a confidence interval (e.g., ...
2
votes
0answers
354 views

Calculation of accuracy (and Cohen's kappa) using sensitivity, specificity, positive and negative predictive values

I read How to calculate specificity from accuracy and sensitivity, but I have two diagnostic performance measures more. Please correct me if I am wrong: if Sensitivity=TP/(TP+FN) Specificity=TN/(TN+...
1
vote
0answers
17 views

Sensitivity and specificity on cases rather than individual days

I have data collected from sensors to detect disease in several individuals. Each individual has a known disease status (labelled Actual in the table below) for ...
0
votes
1answer
259 views

If the position of 0 1 in confusion matrix changes does the formula for sensitivity changes

It may sounds very silly question, but I want to clear myself In confusion matrix we normally takes values like below: ...
1
vote
2answers
284 views

Do you need to calculate sample size to evaluate a new diagnostic test?

I am writing a grant application which will be evaluating a new diagnostic test. The test will predict whether a patient with lung fibrosis will remain stable or progress. I am using an existing ...
1
vote
1answer
163 views

Is it possible for a model to have higher sensitivity/specificity but lower accuracy and AUC?

In the evaluation of classification models, I've found one model to have a higher accuracy and c-statistic (AUC) as compared to a second model. However, the second model has higher sensitivity, ...
1
vote
1answer
118 views

Sensitivity & Specificity calculation

I have a medical diagnostic test $A$ and test $B$. Test $B$ is the current gold standard. Test $A$ has an excellent negative predictive value and if it is negative, test $B$ is not performed as it is ...
-1
votes
1answer
171 views

Is it possible to get high sensitivity but low precision

Consider a case where the number of labelled data as 0 = 1400 and labelled as 1 =100. This dataset is imbalanced with a majority examples belonging to normal class (0) and minority being class labeled ...
0
votes
1answer
251 views

Educational purpose :Calculation of sensitivity and specificity from confusion matrix for this example

Consider a case where the number of labelled data as 0 = 1400 and labelled as 1 =100. The data labelled as 0 denote normal operating conditions and data labelled as ...
12
votes
3answers
1k views

Statistical significance (p-value) for comparing two classifiers with respect to (mean) ROC AUC, sensitivity and specificity

I have a test set of 100 cases and two classifiers. I generated predictions and computed ROC AUC, sensitivity and specificity for both classifiers. Question 1: How can I compute p-value to check if ...
-2
votes
1answer
35 views

how to calculate the specificity, sensitivity and accuracy from the 4 factors

Used decision tree method. Created treemodel for the combined datasets and tried to make prediction on test datasets to calculate error. Please help me on how to calculate the specificity, sensitivity ...
2
votes
1answer
79 views

How to tell the likelihood of getting a false positive? [closed]

Say we have a test for a disease we are comparing with the gold standard. You are given the prevalence of the disease, which is 1%, and sensitivity, 80%, and specificity, 90% and a total population of ...
0
votes
0answers
54 views

Why do dummy regressors decrease sensitivity in linear regression?

For linear regression analysis, I thought that the addition of covariates which are not related to the dependent variable $Y$ does not decrease sensitivity. Such random/dummy regressors can be ...
6
votes
2answers
327 views

Is the sum of p-value and specificity 1

When I look at the definition of p value carefully: $$ p = Pr(X<x|H_0) $$ where $H_0$ means the null hypothesis is true, thus condition negative. That having a test statistic $X<x$ means ...
0
votes
1answer
257 views

What is a good value for Sensitivity and specificity of a predictive model?

Hope all is well. I developed predictive models and I wonder if anyone has some gold standard for sensitivity and specificity from the literature so that I can refer my results to the ones in the ...
3
votes
1answer
209 views

Average of Precision and True negative rate as an accuracy metric

When attempting to construct a classifier for a somewhat imbalanced data set, I was led to the question of measuring the performance of the classifier. One of the first things I thought of was to take ...
0
votes
0answers
257 views

Applying SMOTE and increasing sensitivity

I am trying to analyze lending club data and want to predict whether a loan is risky or safe using random forest with decision tree as a classifier. The data is imbalanced. It contains one-fourth of ...
2
votes
2answers
957 views

Determine baseline for a machine learning project

I work with a unbalanced data set (it is about people who actually bought stuff): Bought stuff: Yes ~ 3% Bought stuff: NO ~97% The most important task for my ...
2
votes
1answer
454 views

Harmonic mean of precision, recall and specificity

I have a system whose performance was characterized in terms of the amount of true positives (TP), true negatives (TN), false positives (FP) and false negatives (FN). There is a rate that summarizes ...
1
vote
0answers
24 views

Compare cut-offs with respect to sensitivity etc

I have a score and a dichotomous criterion for n observations. Using a ROC-Analysis, I found some optimal cut-off, say c. Now, literature (e.g., author of scale with which the score was determined) ...
10
votes
7answers
879 views

Is sensitivity or specificity a function of prevalence?

Standard teaching says that sensitivity and specificity are properties of the test and are independent of prevalence. But isn't this just an assumption? Harrison's principles of internal medicine ...
3
votes
0answers
119 views

Assessing correlated predictions

Let's assume we have a prediction algorithm (if it helps, imagine it's using some boosted tree method) that does daily predictions for whether some event will happen to a unit (e.g. a machine that ...
1
vote
1answer
50 views

What is the variance of negative predictive value when sensitivity is 1?

Variance of NPV is calculated using (1-sensitivity). If estimated( sensitivity) =1 then var(NPV)=0 Is there an alternative formula for var(NPV) similar to Score formula for sensitivity?
2
votes
3answers
235 views

Generalized Sensitivity and Specificity

I'm interested in notation and terminology regarding performance measures related to Sensitivity (SEN) and Specificity (SPC). These two measures are commonly used in binary classification problems. I'...
1
vote
1answer
5k views

What is AUC (Area Under the Curve)?

I've seen many questions posted about AUC but I'm still struggling to understand. I see this definition for AUC everywhere "The AUC is an estimate of the probability that a classifier will rank a ...
2
votes
1answer
351 views

How to determine a sample size

Given that I have an algorithm that classifies data points as 'true' or 'false'. and I want to estimate its FPR, FNR. It is not a supervised model where I start with a large training set of labeled ...
1
vote
1answer
3k views

how to find sensitivity and specificity for more than two levels in the dependent variable from the confusion matrix?

For the above confusion matrix how to compute the sensitivity and specificity manually. I cann't able to understand when the dependent variable has more than two levels
1
vote
1answer
679 views

95% confidence intervals of sensitivity and specificity exceeding 100%. How is it possible?

I read this abstract (unfortunately, I have no access to the full text): https://doi.org/10.1136/bmj.318.7177.193b. How is it possible for 95% confidence intervals of sensitivity and specificity to ...
1
vote
1answer
226 views

How to simulate the distribution of a diagnostic test

I have a parameter described as mean (95% CI low-hi) and would like to build a distribution that will have approximately that mean and lower/upper bounds. The ...
0
votes
0answers
42 views

What is the correct method for comparing sensitivity and specifity of different tests?

What is the correct method of comparing efficiency of different test for one sample of individuals? Are ROC and AUC enough? Comparing of sensitivity and specifity values of the tests with McNemar ...
0
votes
0answers
32 views

How to evaluate sensitivity and specifity of different tests

I am comparing two test using different cut-off values with golden standart test . How is it to be reported. Is it sufficent to report sensitivity, specivity, PPV and NPV with 95 CI or should I ...
1
vote
2answers
2k views

What is the usefulness of detection rate in a confusion matrix?

In the R caret documentation for confusionMatrix(): ...
1
vote
0answers
354 views

Calculating sensitivity and specificity from survival data

I have a diagnostic test performed on 100 participants at baseline. I then follow up these participants for variable periods of time and have data regarding survival. I have used a Cox regression ...
1
vote
1answer
266 views

Sensitivity/Specificity compared to Area Under Curve as measure of Screen Accuracy

In relation to the disorder I'm studying Screen A is reported as having a sensitivity of 90% and a specificity of 89%. Screen B is reported as having a AUC of .79 with no other data provided. Could ...
1
vote
1answer
731 views

Is very high specificity and very low sensitivity attributable to mismatched class sizes?

I've generated a gradient boosting model using r-caret on data which I expect to have little to no predictive value. Class distribution is heavily skewed with ~15000 negative and ~1000 positive. Caret'...