Questions tagged [threshold]
Used (1) for discrete classification (if an instance's predicted probability exceeds a threshold, classify as TRUE, otherwise FALSE), or (2) for discretizing/binning continuous data. *If you are tempted to use this tag, PLEASE read the tag wiki!*
258
questions
0
votes
0
answers
38
views
+50
How to create optimal cut-off scores for a test placing students into different courses
Our goal is to determine optimal cut-off test scores for course placement. The course placement has already been manually assigned to each test-taker. The goal is to replace this manual labor with the ...
- 59
5
votes
1
answer
1k
views
C4.5 How to select the split point (threshold) for a Continuous Attribute
Using the "play golf" or "play ball" data (listed at the bottom), to pick the root node we look at Outlook, Temperature, Humidity, and Wind, to see which has the highest GainRatio.
Now, Outlook will ...
CommunityBot
- 1
0
votes
0
answers
10
views
Finding optimal threshold values between groups using ROC-analyses
I am working on brain tumor data and I would like to be able to separate the uptake of a radioactive tracer in the tumors, depending on the tumor grade. Simplified explanation: Grade 2 tumors have the ...
- 15.9k
2
votes
2
answers
602
views
Bayes factors and ROC curves
The question comes from Kevin Murphy's book, Ch 5, Ex 5.6. Could somebody suggest a solution?
Let $B=p(D|H_1)/p(D|H_0)$ be the bayes factor in favor of model 1. Suppose we plot two ROC curves, one ...
CommunityBot
- 1
0
votes
1
answer
25
views
Are there any difference using scores or probabilities for roc_auc_score and precision_recall_curve functions?
I'm working with a GNN model for link prediction and using precision_recall_curve and roc_auc_score from the ...
- 4,060
0
votes
0
answers
21
views
Can I decrease the sampling interval and still have accurate results?
A company has been collecting water chemistry data annually for 20+ years to monitor water quality. Now they're wondering if they can decrease their sampling interval to once every 2 or 3 years and ...
- 147
0
votes
0
answers
22
views
find the optimal threshold for each class in model.predict (multiclass segmentation)
I have a unet segmentation model, which outputs 5 classes, I would like to find the optimal threshold value for each class using the precision-recall curve:
...
- 1
1
vote
1
answer
312
views
classification ML model: probability of positive label knowing the model score
Question at the intersection of ML and statistics.
I built a binary classification ML model, that for each input observation x outputs the probability p(x) in (0,1) that x belongs to the positive ...
CommunityBot
- 1
2
votes
1
answer
711
views
Picking a model threshold based on Validation set or Test set
I have developed a machine learning model to predict a quantitative output for medical diagnosis (low bone density). I want to convert the model output to a binary outcome and compare it to the gold-...
CommunityBot
- 1
2
votes
1
answer
55
views
ROC curve and thresholds: why does it never have the ideal point at the top left for observations close to certainty?
I am using ROC curves for multi-label classification. I have a classifier that produces a score for each label, say a Logistic Regression that produces a probability. I understand that an ROC curve is ...
- 3,504
3
votes
2
answers
51
views
Prediction of 'other' class
I'm training a MLP classifier with a softmax output that outputs 4 classes.
For my particular application I'd like the classifier to output a fifth 'other' class when the input don't belong to any of ...
- 30.7k
0
votes
0
answers
24
views
What does it mean if optimal classification threshold found on ROC curve is really small?
I've trained a simple NN to perform binary classification with goal of maximizing area under ROC curve. Right now AUC is around 0.85. Out of curiosity, I checked which thresholds are best in terms of ...
- 201
1
vote
1
answer
656
views
Threshold optimization with cross validation
I have an imbalanced dataset; 95% negative class and 5% positive class. I split my data into train (80%) and test (20%) sets. I am using 5-fold cross-validation on the train set to determine the ...
CommunityBot
- 1
0
votes
0
answers
14
views
Setting threshold with dynamic feedback
I have a dataset with 200,000 obs and the following variables:
score (continuous: 0-100), pred (binary: 0/1).
I want to create a binary variabel: pred2, that acts the following:
a) if score is high, ...
- 71.6k
2
votes
1
answer
244
views
ROC curve for comparing probability of default models
I'm trying to compare two different probability of default models together by roc curve.I calculated the PD for 8 company by two different models.I know about the basic of roc curve and i can ...
CommunityBot
- 1
1
vote
1
answer
278
views
Using statistics to determine a trigger/threshold value between continuous variables
I'm working with a few continuous variables, as below:
...
CommunityBot
- 1
0
votes
0
answers
14
views
Regression discontinuity design when only one value is below or above threshold
I conduct a RDD with age as continuous running variable. My threshold is 18 (sharp). I observe people between 17 and 80. Thus I ...
- 51.2k
0
votes
0
answers
28
views
Threshold selection for Peaks over threshold
I have a problem with selecting a threshold for a dataset I cannot show here.
In a MRL-plot, a suitable threshold should be linear with the gradient ξ/(1−ξ) and intercept σ_u/(1−ξ) (as of my ...
- 31
0
votes
0
answers
50
views
How does p-value affect FDR
Is it always true that a lower p-value threshold will lead to a lower FDR?
To answer this question I would say that it is not always true, for example in the case when the distribution is not normal ...
- 101
1
vote
0
answers
93
views
How do you do decision threshold tuning when doing k-fold cross validation?
I'm training a binary classifier for disease detection.
Because of my small amount of data (~1000 datapoints, 10% positive, 90% negative), I've realized that doing an 80-20 train-test split produces ...
- 11
2
votes
1
answer
130
views
Threshold Tuning before or after parameter tuning?
My goal is to increase the F1 score of Class 1 by 1-2%.
I achieved this by changing the threshold from 0.5 to X using the precision recall curve when the dataset is imbalanced. I did this after I have ...
- 111k
0
votes
1
answer
44
views
How does one get from ROC curve to selecting the actual decision threshold of a classification model?
Edit to explain how this is different from the suggested duplicate: Reduce Classification Probability Threshold
My question relates to the same topic, but is thoroughly different, so I'm surprised ...
- 309k
0
votes
0
answers
18
views
Is it possible to derive a threshold when using glmer? [duplicate]
Using the glmer function from the package lme4 I want to know whether and if so, how, it´s possible to obtain a specific threshold with the binomial model?
We fitted the model as follows with ...
- 10.1k
3
votes
2
answers
105
views
When is an unbalanced dataset large enough for calculating a decision threshold?
I have a (large i.e. >1M rows) very unbalanced (1% event label, binary classification) dataset with data from various institutions. At the moment, I train an XGBoost model on this data and get good ...
- 41.1k
0
votes
0
answers
601
views
Plotting precision-recall curve using plot_precision_recall_curve and precision_recall_curve results in different plots
I am plotting the precision-recall curves for my models which I have built using an imbalanced dataset.
I initially plotted the precision-recall curve for my models using the ...
- 133
0
votes
0
answers
18
views
Probability that a linear filtration of a stochastic process exceeds a value for a given number of consecutive samples
I am trying to work out how to calculate the probability that a weighted sum of IID random variables will remain above a given threshold for a particular amount of time ie;
Given a collection of IID ...
- 1
1
vote
0
answers
11
views
How to find thresholds/lower/upper limit for weather factors for species distribution modelling?
What's the best way to find an estimate of weather factors' thresholds/lower/upper limits for a response variable (in my case disease severity) in studies conducted under field conditions? I have ...
- 143
0
votes
0
answers
55
views
Threshold linear regression estimation
When it comes to threshold linear regression, in order to estimate it can we simply divide our dataset according to the threshold rule into 2 datasets and then simply estimate 2 equations with OLS? Or ...
- 165
0
votes
3
answers
673
views
Is it ok a threshold of 0?
I am dealing with a classification problem with a dataset containing 60k rows: 69k are negative class, and 1k is positive.
I trained my models and I obtained the confusion matrices with a threshold of ...
- 31
1
vote
1
answer
414
views
Logistic regression - Does a decision threshold of 0.5 ever make sense?
Say I fit a logistic classifier on a supervised dataset with binary labels.
If I select a threshold of decision of 0.5, which assumption am I implicitly making?
Is there any situation where 0.5 makes ...
- 1,137
1
vote
0
answers
21
views
What is the optimal technique for determining statistical thresholds?
Relevant context: epidemiologists define an outbreak according to six defined stages (investigation, recognition, initiation, acceleration, deceleration, and preparation). From a local perspective, it ...
- 91
0
votes
0
answers
79
views
Model ensembling when classifiers work with different classification thresholds
I have a 2-class classification problem at hand and trained three classifiers to tackle this task. In doing so, I determined for each classifier the optimal classification threshold. For example, ...
- 1
0
votes
1
answer
30
views
Better in AUC and AUC PR, but lower in the optimal threshold
Suppose we have two models; model A and model B.
Model A outperforms both AUC ROC and AUC PR to model B.
However, when we compare the two models with their optimal threshold values, model B ...
- 1,024
1
vote
1
answer
201
views
What happens if we change the threshold probability value for classifying into different class? [duplicate]
Suppose, I classify something as 1 when predicted probability of that event is greater than 0.5 (referred as threshold, henceforth) and 0 when predicted probability of that event is less than 0.5.
...
- 2,184
0
votes
0
answers
59
views
Tune threshold in hyperparameter tuning is giving worse results in MLR
First, I tried not tuning the hyperparameters without setting tune.threshold=TRUE.
...
- 101
82
votes
4
answers
55k
views
Reduce Classification Probability Threshold
I have a question regarding classification in general. Let $f$ be a classifier, which outputs a set of probabilities given some data D. Normally, one would say: well, if $P(c|D) > 0.5$, we will ...
0
votes
1
answer
57
views
What is the procedure to find the optimal decision threshold in an imbalanced classification problem to maximize F1 score?
What is the procedure to find the optimal decision threshold in an imbalanced classification problem to maximize the F1 score? I'm using an xgboost model.
Your help is highly appreciated.
- 111k
0
votes
0
answers
32
views
does the ROC curve of a committee based predictor have any meaning?
would appreciate it if you'd take a moment to read the pipeline I've described below - it relates to how a learner that is based on a committee should be optimized w.r.t the threshold of the ROC curve....
- 125
1
vote
1
answer
41
views
Violation of IID in Peaks over Threshold
I'm using the peaks over threshold method to answer a researchquestion. I'm working with time-series data and the observations are not entirely independent. I know that there is some methods you could ...
- 69
2
votes
1
answer
185
views
Particular sensitivity of random forest accuracy to the decision threshold, but not apparent for other algorithms
I am working on imbalanced dataset. I am usng three algorithms: RF, SVM and J48. Generally an instance is classified as positive if its classification score is greater than 0.5. However, since I am ...
- 51.2k
2
votes
2
answers
955
views
Probability threshold and signal/noise ratio
I'm working on a classification problem in which the underlying signal to identify is very hard to find.
I suppose that this is because the signal/noise ratio is very low.
My questions are ...
- 161
0
votes
0
answers
40
views
Fixing threshold while improving classifier model?
I was playing with a situation where I am forced to fix the threshold of a classifier model. I was wondering if we can fix a cutoff for a mode but add data later on while maintaining the same cutoff? ...
0
votes
0
answers
13
views
Market cointegration
Does anyone here have a code or steps of how to conduct a Threshold vector error correction model? I've been reading a lot of literature but I've only understand the implications of the model. Most of ...
0
votes
0
answers
32
views
Choosing operating point for ~100% recall but max precision on unseen data
I will be training a binary classifier with the goal of getting 100% recall (or, 99.99% recall) when it's in production. The goal is to maximize the precision while keeping the probability of getting ...
- 101
1
vote
1
answer
613
views
How would you find a p threshold for a binary classification prediction? [duplicate]
Lets say that there's a binary classification problem where $X$ ∈ $R_p$ and $Y ∈ \{0,1\} $ and $Pr(Y = 1 | X = x) = p$ for $p$ in $[0,1]$. There is a loss function $L_{falseneg} > 0$ for false ...
- 71.6k
0
votes
0
answers
47
views
How to select a threshold when retraining on the whole dataset?
I am training a Resnet model for a binary classification task, and since my dataset is quite small, I'd like to estimate my performance by doing a 10-fold cross validation, then retrain on the whole ...
4
votes
1
answer
1k
views
Hard thresholding a covariance matrix
I am new to the concept of thresholding a variance-covariance matrix and am having trouble understanding the exact process. I am following Bickel and Levina (2008) in choosing a hard threshold. What ...
- 2,467
1
vote
1
answer
722
views
Threshold Cointegration and Gregory-Hansen Test
I have used the Johansen multivariate cointegration test to see whether a group of ten stock markets are cointegrated. However, I was wondering whether it would be possible to use the Gregory-Hansen ...
CommunityBot
- 1
6
votes
2
answers
838
views
Threshold models and flu epidemic recognition
I'm fooling around with threshold time series models. While I was digging through what others have done, I ran across the CDC's site for flu data.
http://www.cdc.gov/flu/weekly/
About 1/3 of the ...
- 1,118
1
vote
0
answers
43
views
How to find the optimal coefficients of the two predict_proba output matrices of two different classifiers using regression and maximizing accuracy? [closed]
I am performing classification, where there are six labels and two predict_proba (predicted probabilities) matrices as outputs. These two predict_proba matrices correspond to the outputs of two ...
- 11