Questions tagged [threshold]

Used (1) for discrete classification (if an instance's predicted probability exceeds a threshold, classify as TRUE, otherwise FALSE), or (2) for discretizing/binning continuous data. *If you are tempted to use this tag, PLEASE read the tag wiki!*

Filter by
Sorted by
Tagged with
0 votes
0 answers
6 views

What happens if we change the threshold probability value for classifying into different class?

Suppose, I classify something as 1 when predicted probability of that event is greater than 0.5 (referred as threshold, henceforth) and 0 when predicted probability of that event is less than 0.5. ...
0 votes
0 answers
21 views

Tune threshold in hyperparameter tuning is giving worse results in MLR

First, I tried not tuning the hyperparameters without setting tune.threshold=TRUE. ...
1 vote
0 answers
19 views

Prediction of 'other' class

I'm training a MLP classifier with a softmax output that outputs 4 classes. For my particular application I'd like the classifier to output a fifth 'other' class when the input don't belong to any of ...
  • 4,299
0 votes
1 answer
16 views

What is the procedure to find the optimal decision threshold in an imbalanced classification problem to maximize F1 score?

What is the procedure to find the optimal decision threshold in an imbalanced classification problem to maximize the F1 score? I'm using an xgboost model. Your help is highly appreciated.
  • 203
0 votes
0 answers
16 views

does the ROC curve of a committee based predictor have any meaning?

would appreciate it if you'd take a moment to read the pipeline I've described below - it relates to how a learner that is based on a committee should be optimized w.r.t the threshold of the ROC curve....
  • 115
1 vote
1 answer
27 views

Violation of IID in Peaks over Threshold

I'm using the peaks over threshold method to answer a researchquestion. I'm working with time-series data and the observations are not entirely independent. I know that there is some methods you could ...
0 votes
0 answers
17 views

Fixing threshold while improving classifier model?

I was playing with a situation where I am forced to fix the threshold of a classifier model. I was wondering if we can fix a cutoff for a mode but add data later on while maintaining the same cutoff? ...
0 votes
0 answers
11 views

Market cointegration

Does anyone here have a code or steps of how to conduct a Threshold vector error correction model? I've been reading a lot of literature but I've only understand the implications of the model. Most of ...
0 votes
0 answers
14 views

Choosing operating point for ~100% recall but max precision on unseen data

I will be training a binary classifier with the goal of getting 100% recall (or, 99.99% recall) when it's in production. The goal is to maximize the precision while keeping the probability of getting ...
  • 101
0 votes
0 answers
25 views

How to select a threshold when retraining on the whole dataset?

I am training a Resnet model for a binary classification task, and since my dataset is quite small, I'd like to estimate my performance by doing a 10-fold cross validation, then retrain on the whole ...
1 vote
0 answers
39 views

How to find the optimal coefficients of the two predict_proba output matrices of two different classifiers using regression and maximizing accuracy? [closed]

I am performing classification, where there are six labels and two predict_proba (predicted probabilities) matrices as outputs. These two predict_proba matrices correspond to the outputs of two ...
1 vote
1 answer
145 views

Threshold optimization with cross validation

I have an imbalanced dataset; 95% negative class and 5% positive class. I split my data into train (80%) and test (20%) sets. I am using 5-fold cross-validation on the train set to determine the ...
2 votes
1 answer
49 views

Will threshold of logistic regression change accuracy? Any relationship with the incidence of disease? [duplicate]

I am using a logistic regression model to predict breast cancer. I trained and tested the model in a population with a pretty high incidence of breast cancer(since the individuals all went to the ...
0 votes
0 answers
20 views

Predicting behavioural changes based on absolute and change variables

I hope this is the adequate forum for my question. My aim is to predict behavioural changes --which are simply measured by a dummy-- following from changes in some characteristics. To do this, I have ...
  • 1
0 votes
0 answers
43 views

How should I select the optimal threshold using tcplot?

To find the optimal threshold for my gpd model, I made a threshold selection plot for the scale and the shape parameter. But I find it a bit vague how to select the optimal threshold graphically. Can ...
's user avatar
2 votes
0 answers
42 views

Does threshold on the model probability depend upon the spread in the dataset among positive and negative classes (binary classification)?

I think that the threshold on model probability through which one discern positive (y=0) and negative(y=1) class depends on the spread in the training dataset b/w y=0 and y=1. This question came when ...
  • 31
1 vote
0 answers
187 views

XGBoost Feature Importance Changes with Random Seed

Analysis Goal: Identify features that provide an accurate prediction of a binary outcome and also explain how the features are related to the output Data: 72 features and 200 instances. Process: ...
1 vote
0 answers
129 views

Thresholds and Cutoff Values Confusion

I am currently having confusion on a part in the paper: Unal, Ilker. “Defining an Optimal Cut-Point Value in ROC Analysis: An Alternative Approach.” Computational and mathematical methods in medicine ...
0 votes
1 answer
28 views

How to determine a cutoff based on a dataframe with stats (TN TP FN FP MCC F1) on thresholds?

I have gotten a dataframe with corresponding stats (TN TP FN FP MCC F1) on different thresholds (~10,000 thresholds). I'm wondering if there is any statistical methods that help determine the best ...
0 votes
0 answers
30 views

Model ensembling when classifiers work with different classification thresholds

I have a 2-class classification problem at hand and trained three classifiers to tackle this task. In doing so, I determined for each classifier the optimal classification threshold. For example, ...
  • 87
0 votes
0 answers
12 views

Typical magnitude of values for threshold and bias in a NN

I am trying to create a neural network. I want to initialize it with random values for bias and weigths. I could use the whole float or integer range (evenly distributed) one of these, but not evenly ...
  • 101
0 votes
0 answers
15 views

ROC table with integers vs non-integers

Currently I’m using optimal.cutpoints package in R to evaluate ELISA data. I created a data frame from this optimal cut points function that pulled the cut points and their respective sensitivity, ...
1 vote
1 answer
173 views

Normalize different binary prediction probability thresholds

I am trying to build an ensemble of three binary classifiers: A, B and C. Each one generates probabilities for the positive class. My goal is to generate a single probability for each case from the ...
  • 1,657
1 vote
0 answers
145 views

OptimalCutPoints for GLM (Logistic Regression) in R | Find the Threshold

I want to find a threshold value so I can classify which observation is classified to be sucess or not. But I am a confused about how to use the package for logistic regression. For example my glm ...
0 votes
0 answers
9 views

Selecting set of stimuli from norming task

I am a complete newbie, so please bear with me. For my dissertation I applied a norming study to validate corpus coding. I designed my experiment following a Latin square. Sixty participants (10 per ...
  • 1
0 votes
1 answer
61 views

Setting the observation likelihood threshold for outlier detection if you know know the percentage of outliers

Let's assume I have a sensor that gives me measurements $z$ and I know that $50\%$ of the measurements I read are outliers (more than 3 standard deviations away from the real measurement distribution)....
  • 340
1 vote
1 answer
78 views

Relation between AUROC and threshold

As I understand, AUROC tells us the probability the model will score a randomly chosen positive class higher than a randomly chosen negative class. Meaning that, if AUROC = 0.7, than we expect that ...
  • 521
1 vote
0 answers
35 views

When do Efficient Decision Algorithms for 1D Anomaly Detection exist (compared to threshold-tests)?

I'm tasked with investigating whether machine learning algorithms can be used to efficiently identify if a certain type of anomaly is present in the temporal spacing of incoming network packets. ...
  • 11
0 votes
0 answers
20 views

Relation between the lattice points in ROC plot and different pairs of positive and negative classes

Suppose you have a classification problem and you get the following scores from your hypothesis: \begin{bmatrix} 0.87 & 0.30 & 0.40 & 0.10 & 0.23 & 0.70 & 0.90 & 0.60 \end{...
  • 521
0 votes
2 answers
718 views

AUC with different threshold

I know AUC is supposed to be independent on the threshold, which means AUC does not change while the threshold changes. However, I'm getting different AUC values while changing the thresholds. I'm ...
0 votes
0 answers
56 views

thresholding prior to model evaluation

Methodology question. The ML textbook approach is this: perform model fit - optimisation assess fit with Cross-Validation tune decision rule by thresholding on the prediction probability (...
  • 1
0 votes
0 answers
20 views

Multi-class classification: I am applying an uncertainty threshold to send predictions to human, but I want to statistically determine its usefulness

I have the following prediction scenario: $labels\_true = [0,0,0,0,1,1,1,1,2,2,2,2,3,3,3,3]$ $predictions = [0,0,0,1,1,1,1,0,2,2,2,2,3,3,3,3]$ $uncertainty\_in\_prediction = [0.01, 0.01, 0.02, 0.1, 0, ...
1 vote
0 answers
19 views

Random censoring threshold

I am trying to estimate the following censored model: $y_{it}=\beta X_{it} + \epsilon_{it}$. I only observe $y_{it}$ if $y_{it}\leq z_{it}$, otherwise $y_{it}=z_{it}$. The trick is that $z_{it}$ ...
2 votes
1 answer
242 views

Picking a model threshold based on Validation set or Test set

I have developed a machine learning model to predict a quantitative output for medical diagnosis (low bone density). I want to convert the model output to a binary outcome and compare it to the gold-...
  • 35
0 votes
0 answers
5 views

Why does an operational domain isn't an MV-set and why is it a problem for images and computer vision?

I am working on formalizing the operational domain of an algorithm. If the operational domain is the set $D=\{x, g(x)=1\}$ where $g$ is the selector that says when the predictor $f$ can be applied. $g(...
0 votes
0 answers
7 views

Why does the statistical nature of the coverage of an operational domain mathematical formalisation is a problem for images and computer vision?

I am working on formalizing the operational domain of an algorithm. If the operational domain is the set $D=\{x, g(x)=1\}$ where $g$ is the selector that says when the predictor $f$ can be applied. $g(...
2 votes
0 answers
25 views

Define outliers in correlation with right-skewed data (log-log plot)

I have a dataset of counts of occurrences of variables in different classes. For each class, I have an equivalent control created by shuffling the dataset. For instance, this could be words from ...
  • 85
0 votes
0 answers
234 views

Given a multiclass classifier, calculate one threshold per class to maximize recall under precision constraint

Given a classifier $f$, $N$ possible classes, and an input $x$, $f$ produces a class from $[1,..., N]$ and its matching confidence $[0,...,100]$. Then I run $f$ on a large set of examples $X$, and I ...
  • 53
1 vote
0 answers
446 views

Find the best threshold for logistic regression?

I am working on a customer purchase problem. I have 150 campaigns sent by email (or adds if you prefer), that I denote C0, C1 ......
0 votes
0 answers
13 views

Adjusting decision thresholds for dataset with seasonality component

I am working with data on an infection which exhibits seasonality over the year - the disease tends to be more prevalent during the rainy seasons compared with dry seasons. I'm looking at a binary ...
1 vote
1 answer
142 views

Using statistics to determine a trigger/threshold value between continuous variables

I'm working with a few continuous variables, as below: ...
  • 11
0 votes
0 answers
132 views

classification ML model: probability of positive label knowing the model score

Question at the intersection of ML and statistics. I built a binary classification ML model, that for each input observation x outputs the probability p(x) in (0,1) that x belongs to the positive ...
  • 111
1 vote
1 answer
82 views

XGBoost - does it make sense that accuracy decreases as threshold increases?

I'm using XGBoost for a classification problem, and if I need to check how accuracy changes as a function of threshold. As a result, I got that accuracy decreases as the threshold value increases (see ...
  • 137
1 vote
0 answers
137 views

Confirmatory Factor Analysis factor loading cut off threshold

To conduct Confirmatory Factor Analysis (CFA), what is the usual cut off threshold for items' factor loading on their factors? I have found justifications for a more conventional .40 or a more ...
7 votes
3 answers
536 views

Are thresholds for logistic regression models prevalence-specific?

I wonder if thresholds for logistic regression models are prevalence-specific. I assume that they are, however, I am not sure about the basic statistical principles behind it and how to deal with the ...
  • 80
2 votes
0 answers
231 views

Multi-class optimal threshold selection

What are the methods typically used for optimal threshold selection of multiclass classification problem ? I implemented the detector/classifier and generated the confusion matrix, but would like to ...
0 votes
0 answers
196 views

Increasing precision for one label in multiclass classification

I am doing multiclass classification for 3 labell with neural net. The model works fine but when I check precision/recall per label in validation set I can see that precision is a little bit too low ...
1 vote
0 answers
46 views

How to set optimized thresholds to turn a two-class data into a three-class data?

Let's say I have a column, (let us call it column $I$) that has two classes, $[-1,1]$. this column is the prediction of a classification model so it has vectors of probabilities that looks like, for ...
6 votes
1 answer
847 views

Is threshold moving unnecessary in balanced classification problem?

As far as I know, the threshold moving is needed in imbalanced classification problems. The reason why we have to adjust the decision threshold is as follows: Most machine learning algorithms are ...
  • 711
1 vote
0 answers
19 views

Parameter tuning in classification

Recently, I am digging into the selection of tuning parameters in a binary classification. I gathered information by googling and the following is my organization. We can distinct binary ...
  • 711

1
2 3 4 5