Questions tagged [threshold]

Used (1) for discrete classification (if an instance's predicted probability exceeds a threshold, classify as TRUE, otherwise FALSE), or (2) for discretizing/binning continuous data. *If you are tempted to use this tag, PLEASE read the tag wiki!*

Filter by
Sorted by
Tagged with
1 vote
1 answer
36 views
+50

Choosing the correct evaluation metric between F1-score and Area under the Precision-Recall Curve (AUPRC)

We're currently working on detecting specific objects (e.g. poultry farms, hospitals) from satellite images. We've modeled the problem as a binary image classification task (i.e. classifying images ...
meraxes's user avatar
  • 729
2 votes
1 answer
31 views

What is the standard threshold value that is best for accuracy when employing Euclidean distance as a metric for gauging textual similarity?

I'm using Euclidean distance as a metric to compare two sentences for similarity while clustering them using my custom incremental KMeans algorithm. The current threshold value I'm using is 0.7 which ...
sanjay M's user avatar
-1 votes
0 answers
20 views

Why isn't ROC maximum for threshold 0.5?

the way I understand logistic regression, threshold = 0.5 basically produces a hyperplane to classify inputs which minimizes log loss (all of which is converted into a 0 to 1 range using sigmoid), so ...
SRAVAN KOTTA's user avatar
0 votes
0 answers
6 views

finding threshold of activation

Dear crossvalidated community, I am working on a serie of binary measures $y$ (0s and 1s), and an environamental variable $x$ (temperature). I suppose the existence of a $\bar x$ that acts like a &...
Fabio's user avatar
  • 153
2 votes
1 answer
46 views

What if instead re-training our classification model, we only adjust the probability threshold?

As far as i know theoretically our model tend to be drifting/shifting as time goes on and need to be retrained. i wonder if its acceptable that instead of retraining the classification model, we keep ...
raffo's user avatar
  • 23
0 votes
0 answers
13 views

non-linear correlation or finding thresholds for changes with relatively few data points

I'm a novice at stats and thought I'd ask this question to those with much more experience than me. I've got temperature data and count data for number of animals hibernating in man-made boxes. The ...
Bethy's user avatar
  • 1
1 vote
0 answers
42 views

Why following Poisson distribution can be explained as a result of chance?

I'm reading a journal article which applies Poisson distribution in determining how many factors can be regarded as beyond the poverty threshold. My question is: why applying Poisson distribution and ...
ronzenith's user avatar
  • 121
0 votes
0 answers
25 views

Determining inflection point/ threshold values from a Cox regression using cubic splines

I am interested in looking at the impact of time to diagnosis of cancer on death. I will be modelling time to dx as a continuous predictor using cubic splines. I am also interested in determining a ...
user405452's user avatar
0 votes
0 answers
21 views

General threshold-independent statistics

apologies in advance for my bad statistics. I am trying to evaluate two models based on their adverse impact ratio (AIR), which is defined as $$ AIR = \frac{\text{approval rate for protected class}}{\...
stressed's user avatar
  • 113
0 votes
0 answers
43 views

Classification Threshold varies wildly when using ROC curves for threshold moving

I'm trying to do threshold moving to get the appropriate threshold for an imbalanced dataset. I have a 1D timeseries that I am applying a binary transformer-based classifier on. I have: ...
Techie5879's user avatar
1 vote
0 answers
40 views

Threshold choice for Peaks-Over-Threshold

I'm trying to estimate equivalent performances at different events, using Peaks-Over-Threshold from Extreme Value Theory. The challenge is to find the threshold and preferably with same number of ...
Daniel Westergren's user avatar
0 votes
0 answers
17 views

Optimizing Threshold Selection for Improved Sensitivity in Classification Method Without Validated data

Assume we have a datasheet X, this datasheet contains many of samples with different gorup like G1, ...
zhang's user avatar
  • 113
0 votes
2 answers
44 views

Optimal threshlds for continous varibles in order to predict yes/no outcome

I have some trouble finding the best ML approach to solve the following problem: I have a set of continuous variables representing how a specific medical procedure is conducted. I need to find the ...
fb95's user avatar
  • 31
1 vote
1 answer
89 views

Why do we choose thresholds for the logistic regression instead of sampling from a Bernoulli with p (output of the LR) probability?

I would like to know what would be the disadvantages of sampling from a Bernoulli with p probability (p being the output of a logistic regression) to generate the binary classification? Choosing a ...
user11849's user avatar
1 vote
1 answer
36 views

Predicted class probability in threshold moving

I am training a model for the task of Binary classification using H2O.ai. The final output to the user is the probability of class_1. Recently, I found that by ...
Muhammad Ahsan's user avatar
2 votes
0 answers
30 views

Finding the threshold at which events become significantly more frequent

I have two sequences of data: air temperature T and event A. High temperatures can cause Event A, or it can just happen randomly (or other reasons). In the database, some events A are attributed to ...
Indrute's user avatar
  • 21
1 vote
1 answer
125 views

Optimizing a threshold value on a dependent metric using a classifier trained to optimize a threshold-independent metric

Is it a reasonable approach to train a probabilities classifier by optimizing a threshold-independent metric such as AUC, and then using the trained classifier to calibrate the decision threshold ...
Amit S's user avatar
  • 57
0 votes
0 answers
81 views

is a threshold model on ordinal data ~ a link function? SEM OpenMx

Are anyone familiar with OpenMx's capacity for handling ordinal data in SEM using a link function like ordered logit or probit (Stata gsem does this)? Some folks have highlighted issues with feeding ...
Johan's user avatar
  • 145
0 votes
0 answers
14 views

Decision curve analysis - importance of false negatives

I'm implementing a decision curve analysis in my project and since the cost of screening is very low and has the added benefit of reassurance for the patients, false positives aren't of great concern- ...
Wojty's user avatar
  • 75
1 vote
6 answers
344 views

How to create optimal cut-off scores for a test placing students into different courses

Edit: Shared my solution as an answer here Our goal is to determine optimal cut-off test scores for course placement. The course placement has already been manually assigned to each test-taker. The ...
anneirb's user avatar
  • 71
0 votes
2 answers
166 views

Are there any difference using scores or probabilities for roc_auc_score and precision_recall_curve functions?

I'm working with a GNN model for link prediction and using precision_recall_curve and roc_auc_score from the ...
James's user avatar
  • 125
0 votes
0 answers
23 views

Can I decrease the sampling interval and still have accurate results?

A company has been collecting water chemistry data annually for 20+ years to monitor water quality. Now they're wondering if they can decrease their sampling interval to once every 2 or 3 years and ...
Dugan 's user avatar
  • 147
0 votes
0 answers
59 views

find the optimal threshold for each class in model.predict (multiclass segmentation)

I have a unet segmentation model, which outputs 5 classes, I would like to find the optimal threshold value for each class using the precision-recall curve: ...
Krayem67's user avatar
2 votes
1 answer
259 views

ROC curve and thresholds: why does it never have the ideal point at the top left for observations close to certainty?

I am using ROC curves for multi-label classification. I have a classifier that produces a score for each label, say a Logistic Regression that produces a probability. I understand that an ROC curve is ...
emonigma's user avatar
  • 147
0 votes
0 answers
60 views

What does it mean if optimal classification threshold found on ROC curve is really small?

I've trained a simple NN to perform binary classification with goal of maximizing area under ROC curve. Right now AUC is around 0.85. Out of curiosity, I checked which thresholds are best in terms of ...
Brzoskwinia's user avatar
1 vote
0 answers
216 views

How do you do decision threshold tuning when doing k-fold cross validation?

I'm training a binary classifier for disease detection. Because of my small amount of data (~1000 datapoints, 10% positive, 90% negative), I've realized that doing an 80-20 train-test split produces ...
jimbo's user avatar
  • 11
2 votes
1 answer
401 views

Threshold Tuning before or after parameter tuning?

My goal is to increase the F1 score of Class 1 by 1-2%. I achieved this by changing the threshold from 0.5 to X using the precision recall curve when the dataset is imbalanced. I did this after I have ...
Jason Rich Darmawan's user avatar
0 votes
1 answer
108 views

How does one get from ROC curve to selecting the actual decision threshold of a classification model?

Edit to explain how this is different from the suggested duplicate: Reduce Classification Probability Threshold My question relates to the same topic, but is thoroughly different, so I'm surprised ...
NaiveBae's user avatar
  • 257
3 votes
2 answers
134 views

When is an unbalanced dataset large enough for calculating a decision threshold?

I have a (large i.e. >1M rows) very unbalanced (1% event label, binary classification) dataset with data from various institutions. At the moment, I train an XGBoost model on this data and get good ...
Spill4963's user avatar
1 vote
0 answers
16 views

How to find thresholds/lower/upper limit for weather factors for species distribution modelling?

What's the best way to find an estimate of weather factors' thresholds/lower/upper limits for a response variable (in my case disease severity) in studies conducted under field conditions? I have ...
Ahsk's user avatar
  • 405
0 votes
0 answers
1k views

Plotting precision-recall curve using plot_precision_recall_curve and precision_recall_curve results in different plots

I am plotting the precision-recall curves for my models which I have built using an imbalanced dataset. I initially plotted the precision-recall curve for my models using the ...
sums22's user avatar
  • 133
0 votes
0 answers
99 views

Threshold linear regression estimation

When it comes to threshold linear regression, in order to estimate it can we simply divide our dataset according to the threshold rule into 2 datasets and then simply estimate 2 equations with OLS? Or ...
Fatafim's user avatar
  • 195
2 votes
1 answer
835 views

Logistic regression - Does a decision threshold of 0.5 ever make sense?

Say I fit a logistic classifier on a supervised dataset with binary labels. If I select a threshold of decision of 0.5, which assumption am I implicitly making? Is there any situation where 0.5 makes ...
usual me's user avatar
  • 1,227
2 votes
0 answers
22 views

What is the optimal technique for determining statistical thresholds?

Relevant context: epidemiologists define an outbreak according to six defined stages (investigation, recognition, initiation, acceleration, deceleration, and preparation). From a local perspective, it ...
rho's user avatar
  • 101
0 votes
1 answer
39 views

Better in AUC and AUC PR, but lower in the optimal threshold

Suppose we have two models; model A and model B. Model A outperforms both AUC ROC and AUC PR to model B. However, when we compare the two models with their optimal threshold values, model B ...
R and C.F's user avatar
1 vote
1 answer
448 views

What happens if we change the threshold probability value for classifying into different class? [duplicate]

Suppose, I classify something as 1 when predicted probability of that event is greater than 0.5 (referred as threshold, henceforth) and 0 when predicted probability of that event is less than 0.5. ...
stats_geek's user avatar
4 votes
2 answers
132 views

Prediction of 'other' class

I'm training a MLP classifier with a softmax output that outputs 4 classes. For my particular application I'd like the classifier to output a fifth 'other' class when the input don't belong to any of ...
nico's user avatar
  • 4,591
0 votes
1 answer
115 views

What is the procedure to find the optimal decision threshold in an imbalanced classification problem to maximize F1 score?

What is the procedure to find the optimal decision threshold in an imbalanced classification problem to maximize the F1 score? I'm using an xgboost model. Your help is highly appreciated.
NAS_2339's user avatar
  • 223
1 vote
1 answer
51 views

Violation of IID in Peaks over Threshold

I'm using the peaks over threshold method to answer a researchquestion. I'm working with time-series data and the observations are not entirely independent. I know that there is some methods you could ...
alaj1716's user avatar
1 vote
0 answers
45 views

How to find the optimal coefficients of the two predict_proba output matrices of two different classifiers using regression and maximizing accuracy? [closed]

I am performing classification, where there are six labels and two predict_proba (predicted probabilities) matrices as outputs. These two predict_proba matrices correspond to the outputs of two ...
cemrifki's user avatar
1 vote
1 answer
1k views

Threshold optimization with cross validation

I have an imbalanced dataset; 95% negative class and 5% positive class. I split my data into train (80%) and test (20%) sets. I am using 5-fold cross-validation on the train set to determine the ...
modin3956's user avatar
2 votes
1 answer
109 views

Will threshold of logistic regression change accuracy? Any relationship with the incidence of disease? [duplicate]

I am using a logistic regression model to predict breast cancer. I trained and tested the model in a population with a pretty high incidence of breast cancer(since the individuals all went to the ...
user10386405's user avatar
2 votes
0 answers
87 views

Does threshold on the model probability depend upon the spread in the dataset among positive and negative classes (binary classification)?

I think that the threshold on model probability through which one discern positive (y=0) and negative(y=1) class depends on the spread in the training dataset b/w y=0 and y=1. This question came when ...
Mins's user avatar
  • 31
1 vote
0 answers
931 views

XGBoost Feature Importance Changes with Random Seed

Analysis Goal: Identify features that provide an accurate prediction of a binary outcome and also explain how the features are related to the output Data: 72 features and 200 instances. Process: ...
Foolish Frog's user avatar
1 vote
0 answers
699 views

Thresholds and Cutoff Values Confusion

I am currently having confusion on a part in the paper: Unal, Ilker. “Defining an Optimal Cut-Point Value in ROC Analysis: An Alternative Approach.” Computational and mathematical methods in medicine ...
Ramiro Ramirez's user avatar
0 votes
1 answer
62 views

How to determine a cutoff based on a dataframe with stats (TN TP FN FP MCC F1) on thresholds?

I have gotten a dataframe with corresponding stats (TN TP FN FP MCC F1) on different thresholds (~10,000 thresholds). I'm wondering if there is any statistical methods that help determine the best ...
Lin Caijin's user avatar
0 votes
0 answers
99 views

Model ensembling when classifiers work with different classification thresholds

I have a 2-class classification problem at hand and trained three classifiers to tackle this task. In doing so, I determined for each classifier the optimal classification threshold. For example, ...
jonaden's user avatar
  • 87
2 votes
1 answer
684 views

Normalize different binary prediction probability thresholds

I am trying to build an ensemble of three binary classifiers: A, B and C. Each one generates probabilities for the positive class. My goal is to generate a single probability for each case from the ...
Tripartio's user avatar
  • 2,186
2 votes
0 answers
373 views

OptimalCutPoints for GLM (Logistic Regression) in R | Find the Threshold

I want to find a threshold value so I can classify which observation is classified to be sucess or not. But I am a confused about how to use the package for logistic regression. For example my glm ...
Jasmine Helen's user avatar
0 votes
1 answer
142 views

Setting the observation likelihood threshold for outlier detection if you know know the percentage of outliers

Let's assume I have a sensor that gives me measurements $z$ and I know that $50\%$ of the measurements I read are outliers (more than 3 standard deviations away from the real measurement distribution)....
MattSt's user avatar
  • 350

1
2 3 4 5 6