# Questions tagged [threshold]

Used (1) for discrete classification (if an instance's predicted probability exceeds a threshold, classify as TRUE, otherwise FALSE), or (2) for discretizing/binning continuous data. *If you are tempted to use this tag, PLEASE read the tag wiki!*

265 questions
Filter by
Sorted by
Tagged with
10 views

### Is it better to use the W or p-value to determine normality using Shapiro-Wilk?

I have read somewhere that W values above 0.9 are considered normal. So I wanted to use that as a cutoff for normality. However, upon running various scenarios I have come up with the result of W = 0....
23 views

• 113
46 views

### Classification Threshold varies wildly when using ROC curves for threshold moving

I'm trying to do threshold moving to get the appropriate threshold for an imbalanced dataset. I have a 1D timeseries that I am applying a binary transformer-based classifier on. I have: ...
• 101
1 vote
61 views

### Threshold choice for Peaks-Over-Threshold

I'm trying to estimate equivalent performances at different events, using Peaks-Over-Threshold from Extreme Value Theory. The challenge is to find the threshold and preferably with same number of ...
21 views

### Optimizing Threshold Selection for Improved Sensitivity in Classification Method Without Validated data

Assume we have a datasheet X, this datasheet contains many of samples with different gorup like G1, ...
• 113
48 views

### Optimal threshlds for continous varibles in order to predict yes/no outcome

I have some trouble finding the best ML approach to solve the following problem: I have a set of continuous variables representing how a specific medical procedure is conducted. I need to find the ...
• 31
1 vote
113 views

### Why do we choose thresholds for the logistic regression instead of sampling from a Bernoulli with p (output of the LR) probability?

I would like to know what would be the disadvantages of sampling from a Bernoulli with p probability (p being the output of a logistic regression) to generate the binary classification? Choosing a ...
1 vote
46 views

### Predicted class probability in threshold moving

I am training a model for the task of Binary classification using H2O.ai. The final output to the user is the probability of class_1. Recently, I found that by ...
32 views

### Finding the threshold at which events become significantly more frequent

I have two sequences of data: air temperature T and event A. High temperatures can cause Event A, or it can just happen randomly (or other reasons). In the database, some events A are attributed to ...
• 21
1 vote
173 views

### Optimizing a threshold value on a dependent metric using a classifier trained to optimize a threshold-independent metric

Is it a reasonable approach to train a probabilities classifier by optimizing a threshold-independent metric such as AUC, and then using the trained classifier to calibrate the decision threshold ...
• 57
89 views

### is a threshold model on ordinal data ~ a link function? SEM OpenMx

Are anyone familiar with OpenMx's capacity for handling ordinal data in SEM using a link function like ordered logit or probit (Stata gsem does this)? Some folks have highlighted issues with feeding ...
• 145
411 views

### How to create optimal cut-off scores for a test placing students into different courses

Edit: Shared my solution as an answer here Our goal is to determine optimal cut-off test scores for course placement. The course placement has already been manually assigned to each test-taker. The ...
• 61
1 vote
240 views

### Are there any difference using scores or probabilities for roc_auc_score and precision_recall_curve functions?

I'm working with a GNN model for link prediction and using precision_recall_curve and roc_auc_score from the ...
• 135
25 views

### Can I decrease the sampling interval and still have accurate results?

A company has been collecting water chemistry data annually for 20+ years to monitor water quality. Now they're wondering if they can decrease their sampling interval to once every 2 or 3 years and ...
• 157
315 views

### ROC curve and thresholds: why does it never have the ideal point at the top left for observations close to certainty?

I am using ROC curves for multi-label classification. I have a classifier that produces a score for each label, say a Logistic Regression that produces a probability. I understand that an ROC curve is ...
• 167
108 views

### What does it mean if optimal classification threshold found on ROC curve is really small?

I've trained a simple NN to perform binary classification with goal of maximizing area under ROC curve. Right now AUC is around 0.85. Out of curiosity, I checked which thresholds are best in terms of ...
• 291
1 vote
273 views

### How do you do decision threshold tuning when doing k-fold cross validation?

I'm training a binary classifier for disease detection. Because of my small amount of data (~1000 datapoints, 10% positive, 90% negative), I've realized that doing an 80-20 train-test split produces ...
• 11
472 views

### Threshold Tuning before or after parameter tuning?

My goal is to increase the F1 score of Class 1 by 1-2%. I achieved this by changing the threshold from 0.5 to X using the precision recall curve when the dataset is imbalanced. I did this after I have ...
123 views

### How does one get from ROC curve to selecting the actual decision threshold of a classification model?

Edit to explain how this is different from the suggested duplicate: Reduce Classification Probability Threshold My question relates to the same topic, but is thoroughly different, so I'm surprised ...
• 257
137 views

### When is an unbalanced dataset large enough for calculating a decision threshold?

I have a (large i.e. >1M rows) very unbalanced (1% event label, binary classification) dataset with data from various institutions. At the moment, I train an XGBoost model on this data and get good ...
1 vote
19 views

### How to find thresholds/lower/upper limit for weather factors for species distribution modelling?

What's the best way to find an estimate of weather factors' thresholds/lower/upper limits for a response variable (in my case disease severity) in studies conducted under field conditions? I have ...
• 425
1k views

### Plotting precision-recall curve using plot_precision_recall_curve and precision_recall_curve results in different plots

I am plotting the precision-recall curves for my models which I have built using an imbalanced dataset. I initially plotted the precision-recall curve for my models using the ...
• 133
109 views

### Threshold linear regression estimation

When it comes to threshold linear regression, in order to estimate it can we simply divide our dataset according to the threshold rule into 2 datasets and then simply estimate 2 equations with OLS? Or ...
• 205
925 views

### Logistic regression - Does a decision threshold of 0.5 ever make sense?

Say I fit a logistic classifier on a supervised dataset with binary labels. If I select a threshold of decision of 0.5, which assumption am I implicitly making? Is there any situation where 0.5 makes ...
• 1,247
22 views

### What is the optimal technique for determining statistical thresholds?

Relevant context: epidemiologists define an outbreak according to six defined stages (investigation, recognition, initiation, acceleration, deceleration, and preparation). From a local perspective, it ...
• 101
41 views

### Better in AUC and AUC PR, but lower in the optimal threshold

Suppose we have two models; model A and model B. Model A outperforms both AUC ROC and AUC PR to model B. However, when we compare the two models with their optimal threshold values, model B ...
1 vote
500 views

### What happens if we change the threshold probability value for classifying into different class? [duplicate]

Suppose, I classify something as 1 when predicted probability of that event is greater than 0.5 (referred as threshold, henceforth) and 0 when predicted probability of that event is less than 0.5. ...
• 139
208 views

### Prediction of 'other' class

I'm training a MLP classifier with a softmax output that outputs 4 classes. For my particular application I'd like the classifier to output a fifth 'other' class when the input don't belong to any of ...
• 4,591
128 views

### What is the procedure to find the optimal decision threshold in an imbalanced classification problem to maximize F1 score?

What is the procedure to find the optimal decision threshold in an imbalanced classification problem to maximize the F1 score? I'm using an xgboost model. Your help is highly appreciated.
• 223
1 vote
51 views

### Violation of IID in Peaks over Threshold

I'm using the peaks over threshold method to answer a researchquestion. I'm working with time-series data and the observations are not entirely independent. I know that there is some methods you could ...
• 31
1 vote
47 views

### How to find the optimal coefficients of the two predict_proba output matrices of two different classifiers using regression and maximizing accuracy? [closed]

I am performing classification, where there are six labels and two predict_proba (predicted probabilities) matrices as outputs. These two predict_proba matrices correspond to the outputs of two ...
• 11
2k views

### Threshold optimization with cross validation

I have an imbalanced dataset; 95% negative class and 5% positive class. I split my data into train (80%) and test (20%) sets. I am using 5-fold cross-validation on the train set to determine the ...
112 views

### Will threshold of logistic regression change accuracy? Any relationship with the incidence of disease? [duplicate]

I am using a logistic regression model to predict breast cancer. I trained and tested the model in a population with a pretty high incidence of breast cancer(since the individuals all went to the ...
• 521
89 views

### Does threshold on the model probability depend upon the spread in the dataset among positive and negative classes (binary classification)?

I think that the threshold on model probability through which one discern positive (y=0) and negative(y=1) class depends on the spread in the training dataset b/w y=0 and y=1. This question came when ...
• 31
1 vote
977 views

### XGBoost Feature Importance Changes with Random Seed

Analysis Goal: Identify features that provide an accurate prediction of a binary outcome and also explain how the features are related to the output Data: 72 features and 200 instances. Process: ...
1 vote
773 views

### Thresholds and Cutoff Values Confusion

I am currently having confusion on a part in the paper: Unal, Ilker. “Defining an Optimal Cut-Point Value in ROC Analysis: An Alternative Approach.” Computational and mathematical methods in medicine ...
62 views

### How to determine a cutoff based on a dataframe with stats (TN TP FN FP MCC F1) on thresholds?

I have gotten a dataframe with corresponding stats (TN TP FN FP MCC F1) on different thresholds (~10,000 thresholds). I'm wondering if there is any statistical methods that help determine the best ...
• 115
106 views

### Model ensembling when classifiers work with different classification thresholds

I have a 2-class classification problem at hand and trained three classifiers to tackle this task. In doing so, I determined for each classifier the optimal classification threshold. For example, ...
• 87