# Questions tagged [threshold]

Used (1) for discrete classification (if an instance's predicted probability exceeds a threshold, classify as TRUE, otherwise FALSE), or (2) for discretizing/binning continuous data. *If you are tempted to use this tag, PLEASE read the tag wiki!*

258 questions
Filter by
Sorted by
Tagged with
38 views
+50

### How to create optimal cut-off scores for a test placing students into different courses

Our goal is to determine optimal cut-off test scores for course placement. The course placement has already been manually assigned to each test-taker. The goal is to replace this manual labor with the ...
1k views

### C4.5 How to select the split point (threshold) for a Continuous Attribute

Using the "play golf" or "play ball" data (listed at the bottom), to pick the root node we look at Outlook, Temperature, Humidity, and Wind, to see which has the highest GainRatio. Now, Outlook will ...
10 views

### Finding optimal threshold values between groups using ROC-analyses

I am working on brain tumor data and I would like to be able to separate the uptake of a radioactive tracer in the tumors, depending on the tumor grade. Simplified explanation: Grade 2 tumors have the ...
602 views

### Bayes factors and ROC curves

The question comes from Kevin Murphy's book, Ch 5, Ex 5.6. Could somebody suggest a solution? Let $B=p(D|H_1)/p(D|H_0)$ be the bayes factor in favor of model 1. Suppose we plot two ROC curves, one ...
25 views

### Are there any difference using scores or probabilities for roc_auc_score and precision_recall_curve functions?

I'm working with a GNN model for link prediction and using precision_recall_curve and roc_auc_score from the ...
21 views

### Can I decrease the sampling interval and still have accurate results?

A company has been collecting water chemistry data annually for 20+ years to monitor water quality. Now they're wondering if they can decrease their sampling interval to once every 2 or 3 years and ...
22 views

### find the optimal threshold for each class in model.predict (multiclass segmentation)

I have a unet segmentation model, which outputs 5 classes, I would like to find the optimal threshold value for each class using the precision-recall curve: ...
1 vote
312 views

### classification ML model: probability of positive label knowing the model score

Question at the intersection of ML and statistics. I built a binary classification ML model, that for each input observation x outputs the probability p(x) in (0,1) that x belongs to the positive ...
711 views

### Picking a model threshold based on Validation set or Test set

I have developed a machine learning model to predict a quantitative output for medical diagnosis (low bone density). I want to convert the model output to a binary outcome and compare it to the gold-...
55 views

### ROC curve and thresholds: why does it never have the ideal point at the top left for observations close to certainty?

I am using ROC curves for multi-label classification. I have a classifier that produces a score for each label, say a Logistic Regression that produces a probability. I understand that an ROC curve is ...
51 views

### Prediction of 'other' class

I'm training a MLP classifier with a softmax output that outputs 4 classes. For my particular application I'd like the classifier to output a fifth 'other' class when the input don't belong to any of ...
24 views

### What does it mean if optimal classification threshold found on ROC curve is really small?

I've trained a simple NN to perform binary classification with goal of maximizing area under ROC curve. Right now AUC is around 0.85. Out of curiosity, I checked which thresholds are best in terms of ...
1 vote
656 views

### Threshold optimization with cross validation

I have an imbalanced dataset; 95% negative class and 5% positive class. I split my data into train (80%) and test (20%) sets. I am using 5-fold cross-validation on the train set to determine the ...
14 views

### Setting threshold with dynamic feedback

I have a dataset with 200,000 obs and the following variables: score (continuous: 0-100), pred (binary: 0/1). I want to create a binary variabel: pred2, that acts the following: a) if score is high, ...
244 views

### ROC curve for comparing probability of default models

I'm trying to compare two different probability of default models together by roc curve.I calculated the PD for 8 company by two different models.I know about the basic of roc curve and i can ...
1 vote
278 views

### Using statistics to determine a trigger/threshold value between continuous variables

I'm working with a few continuous variables, as below: ...
14 views

### Regression discontinuity design when only one value is below or above threshold

I conduct a RDD with age as continuous running variable. My threshold is 18 (sharp). I observe people between 17 and 80. Thus I ...
28 views

### Threshold selection for Peaks over threshold

I have a problem with selecting a threshold for a dataset I cannot show here. In a MRL-plot, a suitable threshold should be linear with the gradient ξ/(1−ξ) and intercept σ_u/(1−ξ) (as of my ...
50 views

### How does p-value affect FDR

Is it always true that a lower p-value threshold will lead to a lower FDR? To answer this question I would say that it is not always true, for example in the case when the distribution is not normal ...
1 vote
93 views

### How do you do decision threshold tuning when doing k-fold cross validation?

I'm training a binary classifier for disease detection. Because of my small amount of data (~1000 datapoints, 10% positive, 90% negative), I've realized that doing an 80-20 train-test split produces ...
130 views

### Threshold Tuning before or after parameter tuning?

My goal is to increase the F1 score of Class 1 by 1-2%. I achieved this by changing the threshold from 0.5 to X using the precision recall curve when the dataset is imbalanced. I did this after I have ...
44 views

### How does one get from ROC curve to selecting the actual decision threshold of a classification model?

Edit to explain how this is different from the suggested duplicate: Reduce Classification Probability Threshold My question relates to the same topic, but is thoroughly different, so I'm surprised ...
18 views

### Is it possible to derive a threshold when using glmer? [duplicate]

Using the glmer function from the package lme4 I want to know whether and if so, how, it´s possible to obtain a specific threshold with the binomial model? We fitted the model as follows with ...
105 views

### When is an unbalanced dataset large enough for calculating a decision threshold?

I have a (large i.e. >1M rows) very unbalanced (1% event label, binary classification) dataset with data from various institutions. At the moment, I train an XGBoost model on this data and get good ...
601 views

### Plotting precision-recall curve using plot_precision_recall_curve and precision_recall_curve results in different plots

I am plotting the precision-recall curves for my models which I have built using an imbalanced dataset. I initially plotted the precision-recall curve for my models using the ...
18 views

### Probability that a linear filtration of a stochastic process exceeds a value for a given number of consecutive samples

I am trying to work out how to calculate the probability that a weighted sum of IID random variables will remain above a given threshold for a particular amount of time ie; Given a collection of IID ...
1 vote
11 views

### How to find thresholds/lower/upper limit for weather factors for species distribution modelling?

What's the best way to find an estimate of weather factors' thresholds/lower/upper limits for a response variable (in my case disease severity) in studies conducted under field conditions? I have ...
55 views

### Threshold linear regression estimation

When it comes to threshold linear regression, in order to estimate it can we simply divide our dataset according to the threshold rule into 2 datasets and then simply estimate 2 equations with OLS? Or ...
673 views

### Is it ok a threshold of 0?

I am dealing with a classification problem with a dataset containing 60k rows: 69k are negative class, and 1k is positive. I trained my models and I obtained the confusion matrices with a threshold of ...
1 vote
414 views

### Logistic regression - Does a decision threshold of 0.5 ever make sense?

Say I fit a logistic classifier on a supervised dataset with binary labels. If I select a threshold of decision of 0.5, which assumption am I implicitly making? Is there any situation where 0.5 makes ...
1 vote
21 views

### What is the optimal technique for determining statistical thresholds?

Relevant context: epidemiologists define an outbreak according to six defined stages (investigation, recognition, initiation, acceleration, deceleration, and preparation). From a local perspective, it ...
79 views

### Model ensembling when classifiers work with different classification thresholds

I have a 2-class classification problem at hand and trained three classifiers to tackle this task. In doing so, I determined for each classifier the optimal classification threshold. For example, ...
30 views

### Better in AUC and AUC PR, but lower in the optimal threshold

Suppose we have two models; model A and model B. Model A outperforms both AUC ROC and AUC PR to model B. However, when we compare the two models with their optimal threshold values, model B ...
1 vote
201 views

### What happens if we change the threshold probability value for classifying into different class? [duplicate]

Suppose, I classify something as 1 when predicted probability of that event is greater than 0.5 (referred as threshold, henceforth) and 0 when predicted probability of that event is less than 0.5. ...
59 views

### Tune threshold in hyperparameter tuning is giving worse results in MLR

First, I tried not tuning the hyperparameters without setting tune.threshold=TRUE. ...
55k views

### Reduce Classification Probability Threshold

I have a question regarding classification in general. Let $f$ be a classifier, which outputs a set of probabilities given some data D. Normally, one would say: well, if $P(c|D) > 0.5$, we will ...
57 views

### What is the procedure to find the optimal decision threshold in an imbalanced classification problem to maximize F1 score?

What is the procedure to find the optimal decision threshold in an imbalanced classification problem to maximize the F1 score? I'm using an xgboost model. Your help is highly appreciated.
32 views

### does the ROC curve of a committee based predictor have any meaning?

would appreciate it if you'd take a moment to read the pipeline I've described below - it relates to how a learner that is based on a committee should be optimized w.r.t the threshold of the ROC curve....
1 vote
41 views

### Violation of IID in Peaks over Threshold

I'm using the peaks over threshold method to answer a researchquestion. I'm working with time-series data and the observations are not entirely independent. I know that there is some methods you could ...
185 views

### Particular sensitivity of random forest accuracy to the decision threshold, but not apparent for other algorithms

I am working on imbalanced dataset. I am usng three algorithms: RF, SVM and J48. Generally an instance is classified as positive if its classification score is greater than 0.5. However, since I am ...
955 views

### Probability threshold and signal/noise ratio

I'm working on a classification problem in which the underlying signal to identify is very hard to find. I suppose that this is because the signal/noise ratio is very low. My questions are ...
40 views

### Fixing threshold while improving classifier model?

I was playing with a situation where I am forced to fix the threshold of a classifier model. I was wondering if we can fix a cutoff for a mode but add data later on while maintaining the same cutoff? ...
13 views

### Market cointegration

Does anyone here have a code or steps of how to conduct a Threshold vector error correction model? I've been reading a lot of literature but I've only understand the implications of the model. Most of ...
32 views

### Choosing operating point for ~100% recall but max precision on unseen data

I will be training a binary classifier with the goal of getting 100% recall (or, 99.99% recall) when it's in production. The goal is to maximize the precision while keeping the probability of getting ...
1 vote
613 views

### How would you find a p threshold for a binary classification prediction? [duplicate]

Lets say that there's a binary classification problem where $X$ ∈ $R_p$ and $Y ∈ \{0,1\}$ and $Pr(Y = 1 | X = x) = p$ for $p$ in $[0,1]$. There is a loss function $L_{falseneg} > 0$ for false ...
47 views

### How to select a threshold when retraining on the whole dataset?

I am training a Resnet model for a binary classification task, and since my dataset is quite small, I'd like to estimate my performance by doing a 10-fold cross validation, then retrain on the whole ...
1k views

### Hard thresholding a covariance matrix

I am new to the concept of thresholding a variance-covariance matrix and am having trouble understanding the exact process. I am following Bickel and Levina (2008) in choosing a hard threshold. What ...
1 vote
722 views

### Threshold Cointegration and Gregory-Hansen Test

I have used the Johansen multivariate cointegration test to see whether a group of ten stock markets are cointegrated. However, I was wondering whether it would be possible to use the Gregory-Hansen ...