Questions tagged [false-positive-rate]
In a test for a condition (such as a disease) the false positive rate is the proportion of subjects incorrectly classified as having the condition.
56
questions
0
votes
0
answers
18
views
What does positive likelihood ratio mean outside of medicine?
I am a meteorologist and I regularly hear of POD (we call it probability of detection) $ POD = \frac{TP}{TP+FN} $ as well as FAR (False Alarm Rate) $ FAR = \frac{FP}{FP+TN}$. But I recently had a ...
- 123
0
votes
0
answers
19
views
False positive correction categorical data
I am trying to understand how to correct for chance categorisation as either impaired/not impaired across a range of measures.
Briefly, I am looking to summarise prevalence statistics across ~40 ...
- 23
1
vote
0
answers
18
views
PPV with two tests
There are two tests for a disease. Its prevalence is equal to 0.5. Test A has a sensitivity of 0.7 and a specificity of 0.7. Test B has a sensitivity of 0.8 and a specificity of 0.9. Assume that both ...
user378740
1
vote
1
answer
16
views
Create dataset by sampling from near-boundary of binary classifier to improve accuracy
Say I have some binary classifier $f: X \to [0, 1]$. I think the following bi-stage training method is straightforward to reduce error.
Step1. Sample uniformly from $X$ and create dataset with ...
- 191
0
votes
0
answers
17
views
Performance metrics for time series anomaly detection with rolling horizon
I have a time series problem, where I am trying to find indication of break-down of machines in a factory based on a set of features related to said machines and their function (e.g. hours working, ...
- 1
0
votes
0
answers
66
views
How to reduce the number of false positives in object detection in fusion pipeline?
I am training a sequential fusion network using first in image and then in LiDAR point cloud. Specifically, I tend to using the result of image detection result, to improve the performance for far ...
- 1
0
votes
0
answers
20
views
Harmonic mean of false positive and false discovery rates (analogous to F1)
F1 is the harmonic mean of recall (aka sensitivity, or true positive rate, TPR) and precision (aka positive predictive value, PPV).
$\text{TPR} = \text{Pr(predicted:Pos | Pos)} =$ TP/P (wikipedia ...
- 101
0
votes
0
answers
35
views
Calculate probability of alarm and the posterior probability of this alarm being false over different frequencies of output
I have the following information for an automatic detection system that output a warning when a signal is detected:
Specificity: .99 (i.e. a false positive rate $FP = .01$)
Sensitivity: .9 (i.e. a ...
- 1
1
vote
1
answer
21
views
If AUC is 1 can be FNR 0.03?
If AUC (Area under Curve) is 1, do we expect FNR (False Negative Rate) be 0?
For example, can FNR be 0.03?
- 221
1
vote
0
answers
133
views
How to interpret ROC curve if my FPR (X-axis) values ranges from 0 to 0.02?
I am plotting the ROC curve for my classification problem. The results I am getting for the problem are TPR ranges from 0 to 1, but the FPR ranges from 0 to 0.02. I have plotted the ROC curve by ...
- 33
1
vote
0
answers
15
views
Can a low rate of true positives minimise false negatives?
I am analysing thousands of scientific pictures which all show a similar light pattern, trying to identify local anomalies on the pictures. As it is nearly impossible to inspect each single image due ...
- 11
0
votes
0
answers
26
views
Does contamination in the training dataset change the distribution of the predicted probability of the model?
While using a not regularized loss function, oversampled simply binary classification dataset, I observed that whenever I am adding correctly classified examples, there is no change in the ...
- 31
2
votes
0
answers
57
views
Does threshold on the model probability depend upon the spread in the dataset among positive and negative classes (binary classification)?
I think that the threshold on model probability through which one discern positive (y=0) and negative(y=1) class depends on the spread in the training dataset b/w y=0 and y=1.
This question came when ...
- 31
3
votes
2
answers
1k
views
why I am getting TP and FP in confusion matrix is 0, how to get it right?
why I am getting TP and FP in confusion matrix is 0, how to get it right?
tp = 0,fp = 0,tn = 9847,fn = 18
5
votes
2
answers
85
views
When using ROC curves for WWII Radars, what was the TN?
One of the origins of ROC curves seems to be to compare radar systems in WWII (source). How did they actually compute the False Positive Rate when they didn't have an estimate for True Negatives?
If I ...
- 53
0
votes
1
answer
34
views
False and true positives in research
Let's suppose that there's a new apple tree disease, and there exists a test for the disease that has a false positive rate of 5%, the test produces no false negatives (if an apple tree has the ...
- 5
2
votes
1
answer
2k
views
Relationship between Recall, TPR, FPR and Precision
Can Precision and Recall be used to Generate TPR or FPR? In other words, is there any formula that relates the following Evaluation metrics?
True Positive Rate (TPR) with either Precision or Recall (...
- 21
0
votes
0
answers
58
views
Analytic expression for false-negative rate of binomial tests?
I wrote a previous question yesterday which was maybe too long and boring to read. So to try to get an answer, I've boiled down my question to something short and specific which is:
Is there an ...
- 385
6
votes
3
answers
847
views
How many coin flips are needed to reliably know a coin of weight w is unfair?
I want to find out how many flips I need to flip a coin to reliably know that it is an unfair coin.
The issue is that as the coin becomes closer to 50/50, the more false-negatives you will have if you ...
- 385
3
votes
1
answer
133
views
Is Partial Correlation useful for noisy data?
Explaining The Problem
Important question in data analysis is testing observed relationships for confounding factors. Partial Correlation is a metric designed to do specifically that. The general idea ...
- 1,749
0
votes
0
answers
2k
views
Reduce false positive in extremely high imbalance testing set
I have built a CNN model to classify positive and negative in my data, the accuracy is around 85% with FPR is 16%. I know the FPR is high but it gives an acceptable number of FP in training and ...
- 1
3
votes
2
answers
217
views
ROC curve from an array of Confusion Matrices (true positive rates and false positive rates)
How can we create an ROC curve from an array of Confusion Matrices (true positive rates and false positive rates)?
- 1,346
10
votes
2
answers
13k
views
Best way to reduce false positive of binary classification to exactly 0?
I'm working on a task that even a 0.00001 fp rate is not acceptable, because detecting something as a positive when its not will have very bad consequences in this task, so it needs to be exactly 0 ...
- 245
2
votes
1
answer
339
views
Formula for expected false positive & negative rates in hiring decisions based on r
I would like to look at the size of the expected false positive and false negative rates in employment hiring decisions. Let's assume that it is useful to dichotomize job performance after hiring.
...
- 3,116
4
votes
2
answers
598
views
Does up-sampling lead to lots of false positives in production?
Say we have a dataset with a binary outcome variable that takes the positive case (outcome = 1) roughly 20% of the time. Often, we would modify the training set by ...
- 173
1
vote
0
answers
40
views
Sensitivity and specificity with multiple sequential tests
This has been on my mind for a while, and I would appreciate a statistician's view on the matter. Suppose that a given test has sensitivity $a$ and specificity $b$. Now, periodically apply this test ...
3
votes
1
answer
149
views
Estimating positive and negative predictive value without knowing the prevalence
There is a lot of discussion about the positive predictive value of a test currently. I know that if I know specificity, sensitivity of a test and the prevalence $p$ in the sample, then I can easily ...
- 2,469
0
votes
0
answers
85
views
Correlation between two non-independent samples (dichotomized data / multiple testing)
I've been looking through text books but unable to find a precise answer to this question, but it seems important so maybe I'm looking in wrong places. Imagine some population with a normal ...
- 261
0
votes
0
answers
68
views
How do you cope with the risk of false-positives in exploratory analysis?
Let's say that I'm running exploratory analysis on a dataset.
For instance, let's say that the dataset consists of several features and two groups and I want to see which features are significantly ...
user289381
2
votes
2
answers
76
views
Aren't all tests sensitive to the prevalence of a disease in the population?
I'm trying to understand the difference between the false-positive rates of two kinds of COVID-19 tests: PCR and antibody.
The former indicates if someone is currently sick. The latter indicates if ...
- 161
0
votes
0
answers
749
views
How can I fix type 1 error for my logistic regression model?
I am testing if the sound pressure levels(rms) of shipping noise affect the presence or absence of the fin whales by acoustic monitoring for 4 months of data from one location. Here my response ...
1
vote
1
answer
114
views
How to bias against false positives given estimate and standard deviation?
For 1000 newsletter recipients, I crudely estimate the likelihood (p) of them reading the next email sent as:
...
- 301
0
votes
1
answer
113
views
If a false positive rate of α is desired, what is the acceptable range of false positives one would expect from a sample size of n?
For instance, suppose I design a test to have a false positive rate of 5%. I am going to perform this test on what is assumed to be an all negative population. If my sample size was 20, I would expect ...
- 101
1
vote
0
answers
20
views
Procedure to Smooth Noise for Threshold Values
Suppose I am looking at an estimate $\hat\beta$ in a clinical trial data.
With $a=.05$, a patient with $\hat\beta>1$ is considered to have some medical condition.
A daily example could be a 24-hr ...
- 621
2
votes
1
answer
769
views
Base rate of accuracy after resampling for classification problems
If I had an imbalanced dataset with 10% positive instances and 90% negative ones, the base rate for accuracy before resampling is 90%.
But what about I resampled the data such that I have an equal ...
- 23
1
vote
0
answers
115
views
Deriving a loss function properly accounting for different error type costs
Consider a classification setting like a medical test: Not finding an existing health issue might be much worse (by a factor of 50) than assuming an issue when there is none.
I.e. a setting in which ...
- 161
2
votes
1
answer
103
views
What is "positive"?
I'm deeply confused by some concepts.
We often hear the term true/false positive/negative.
While it is straightforward to tell if the result is true or false, I ...
- 153
1
vote
2
answers
386
views
FPR in Confusion Matrix
I was trying to manually calculate $\text{TPR}$ and $\text{FPR}$ for the given data. But unfortunately I dont have any false positive cases in my dataset and even no true positive cases. So I am ...
0
votes
0
answers
245
views
Oversampling/Undersampling in respect to Train and Test - Isolation Forest
I've got a quite imbalanced data set.
144.496 : 162 -> ratio of 1000:1
I would like to use IsolationForest to detect the 162 anomalys.
I've already split the data.
However, the iForest doesn't ...
- 1
2
votes
1
answer
484
views
False positive/negative rate in ridge and lasso regressions
I have a confusion matrix of true and estimated $\boldsymbol{\beta}$ vectors of lasso and ridge models from a replicate of a simulation study, say. The following tables illustrate the scenario.
$$\...
- 283
0
votes
2
answers
186
views
How to make sense of both high FNR and NPV?
So, even though this is one of the most basic and already explained things about statistics I always seem to find way not to get it.
Having this table which shows high Negative Predictive Value (352/...
- 1
1
vote
1
answer
155
views
SVM probability output threshold as (1 - FPR)?
I have a binary SVM with probability output (via Platt scaling). I want to set a threshold on the probability outputs since I want to trade off making false positives/negatives. Is it possible to ...
- 93
2
votes
0
answers
88
views
How to interpret a False Discovery Rate plot
It is the first time that I am trying to calculate the FDR and I use the fdrtool package in R. I want both, local and tail area graphs and I think the third ...
- 21
0
votes
0
answers
13
views
Estimating false accept rates from imposter scores below a threshold
I have a system that compares two items and produces a match score. Scores below a threshold are manually inspected to determine if they match or don't(imposter). Scores above the threshold are ...
- 1
2
votes
1
answer
150
views
Is most published research indeed false?
I have come across (Ioannidis, 2005) which explains several reasons (mainly statistics-related, that's why I post this question here) to justify the claim that most published research is indeed false. ...
- 2,500
0
votes
1
answer
37
views
Optimal solution for false positive effect on business strategy
I am facing a challenge to reach an optimal solution. I will try to explain with an example :
Suppose I have created an algorithm to predict if a customer will subscribe to bank deposit plan or not. ...
- 1
0
votes
1
answer
204
views
Is it possible to estimate accuracy, precision and recall with the given data?
Background: I talked to my friend today and according to herm(him/her) I can calculate precision, recall and accuracy with the current information.
Total instances T: 19,532.
Instances belonging to ...
- 171
1
vote
0
answers
38
views
Exclude areas of feature space without getting false negatives
I am using a decision tree classifier to split the feature space according to two classes ( A and B).
Events of class A are important and I want to classify all of them correct, i.e. no false ...
- 11
0
votes
0
answers
27
views
Estimating true positive rate with a confidence interval given a classifier + unknown dataset truths
Let's say I've built a binary classifier - one that for instance, can classify whether a particular transaction is fraudulent or not. This classifier outputs a 1 or a 0, with a given degree of ...
- 101
2
votes
2
answers
549
views
Is it the case that β+power=1?
In psychology papers that do prospective power analyses (example), one often notices the convention of assuming β values (false negative rates) of 0.20, and power levels of 0.80. In other words (if I ...
- 2,152