Questions tagged [threshold]

Used (1) for discrete classification (if an instance's predicted probability exceeds a threshold, classify as TRUE, otherwise FALSE), or (2) for discretizing/binning continuous data. *If you are tempted to use this tag, PLEASE read the tag wiki!*

Filter by
Sorted by
Tagged with
1 vote
0 answers
13 views

How to find the optimal coefficients of the two predict_proba output matrices of two different classifiers using regression and maximizing accuracy?

I am performing classification, where there are six labels and two predict_proba (predicted probabilities) matrices as outputs. These two predict_proba matrices correspond to the outputs of two ...
user avatar
1 vote
1 answer
18 views

Threshold optimization with cross validation

I have an imbalanced dataset; 95% negative class and 5% positive class. I split my data into train (80%) and test (20%) sets. I am using 5-fold cross-validation on the train set to determine the ...
user avatar
2 votes
1 answer
37 views

Will threshold of logistic regression change accuracy? Any relationship with the incidence of disease? [duplicate]

I am using a logistic regression model to predict breast cancer. I trained and tested the model in a population with a pretty high incidence of breast cancer(since the individuals all went to the ...
user avatar
0 votes
0 answers
18 views

Predicting behavioural changes based on absolute and change variables

I hope this is the adequate forum for my question. My aim is to predict behavioural changes --which are simply measured by a dummy-- following from changes in some characteristics. To do this, I have ...
user avatar
  • 1
0 votes
0 answers
14 views

How should I select the optimal threshold using tcplot?

To find the optimal threshold for my gpd model, I made a threshold selection plot for the scale and the shape parameter. But I find it a bit vague how to select the optimal threshold graphically. Can ...
user avatar
1 vote
0 answers
20 views

Does threshold on the model probability depend upon the spread in the dataset among positive and negative classes (binary classification)?

I think that the threshold on model probability through which one discern positive (y=0) and negative(y=1) class depends on the spread in the training dataset b/w y=0 and y=1. This question came when ...
user avatar
  • 21
1 vote
0 answers
40 views

XGBoost Feature Importance Changes with Random Seed

Analysis Goal: Identify features that provide an accurate prediction of a binary outcome and also explain how the features are related to the output Data: 72 features and 200 instances. Process: ...
user avatar
1 vote
0 answers
33 views

Thresholds and Cutoff Values Confusion

I am currently having confusion on a part in the paper: Unal, Ilker. “Defining an Optimal Cut-Point Value in ROC Analysis: An Alternative Approach.” Computational and mathematical methods in medicine ...
user avatar
0 votes
1 answer
16 views

How to determine a cutoff based on a dataframe with stats (TN TP FN FP MCC F1) on thresholds?

I have gotten a dataframe with corresponding stats (TN TP FN FP MCC F1) on different thresholds (~10,000 thresholds). I'm wondering if there is any statistical methods that help determine the best ...
user avatar
0 votes
0 answers
15 views

Model ensembling when classifiers work with different classification thresholds

I have a 2-class classification problem at hand and trained three classifiers to tackle this task. In doing so, I determined for each classifier the optimal classification threshold. For example, ...
user avatar
  • 87
0 votes
0 answers
8 views

Typical magnitude of values for threshold and bias in a NN

I am trying to create a neural network. I want to initialize it with random values for bias and weigths. I could use the whole float or integer range (evenly distributed) one of these, but not evenly ...
user avatar
  • 101
0 votes
0 answers
15 views

ROC table with integers vs non-integers

Currently I’m using optimal.cutpoints package in R to evaluate ELISA data. I created a data frame from this optimal cut points function that pulled the cut points and their respective sensitivity, ...
user avatar
1 vote
1 answer
73 views

Normalize different binary prediction probability thresholds

I am trying to build an ensemble of three binary classifiers: A, B and C. Each one generates probabilities for the positive class. My goal is to generate a single probability for each case from the ...
user avatar
  • 1,557
1 vote
0 answers
85 views

OptimalCutPoints for GLM (Logistic Regression) in R | Find the Threshold

I want to find a threshold value so I can classify which observation is classified to be sucess or not. But I am a confused about how to use the package for logistic regression. For example my glm ...
user avatar
0 votes
0 answers
9 views

Selecting set of stimuli from norming task

I am a complete newbie, so please bear with me. For my dissertation I applied a norming study to validate corpus coding. I designed my experiment following a Latin square. Sixty participants (10 per ...
user avatar
  • 1
0 votes
1 answer
30 views

Setting the observation likelihood threshold for outlier detection if you know know the percentage of outliers

Let's assume I have a sensor that gives me measurements $z$ and I know that $50\%$ of the measurements I read are outliers (more than 3 standard deviations away from the real measurement distribution)....
user avatar
  • 340
1 vote
1 answer
47 views

Relation between AUROC and threshold

As I understand, AUROC tells us the probability the model will score a randomly chosen positive class higher than a randomly chosen negative class. Meaning that, if AUROC = 0.7, than we expect that ...
user avatar
  • 481
1 vote
0 answers
33 views

When do Efficient Decision Algorithms for 1D Anomaly Detection exist (compared to threshold-tests)?

I'm tasked with investigating whether machine learning algorithms can be used to efficiently identify if a certain type of anomaly is present in the temporal spacing of incoming network packets. ...
user avatar
  • 11
0 votes
0 answers
19 views

Relation between the lattice points in ROC plot and different pairs of positive and negative classes

Suppose you have a classification problem and you get the following scores from your hypothesis: \begin{bmatrix} 0.87 & 0.30 & 0.40 & 0.10 & 0.23 & 0.70 & 0.90 & 0.60 \end{...
user avatar
  • 481
0 votes
2 answers
200 views

AUC with different threshold

I know AUC is supposed to be independent on the threshold, which means AUC does not change while the threshold changes. However, I'm getting different AUC values while changing the thresholds. I'm ...
user avatar
0 votes
0 answers
50 views

thresholding prior to model evaluation

Methodology question. The ML textbook approach is this: perform model fit - optimisation assess fit with Cross-Validation tune decision rule by thresholding on the prediction probability (...
user avatar
  • 1
0 votes
0 answers
17 views

Multi-class classification: I am applying an uncertainty threshold to send predictions to human, but I want to statistically determine its usefulness

I have the following prediction scenario: $labels\_true = [0,0,0,0,1,1,1,1,2,2,2,2,3,3,3,3]$ $predictions = [0,0,0,1,1,1,1,0,2,2,2,2,3,3,3,3]$ $uncertainty\_in\_prediction = [0.01, 0.01, 0.02, 0.1, 0, ...
user avatar
1 vote
0 answers
19 views

Random censoring threshold

I am trying to estimate the following censored model: $y_{it}=\beta X_{it} + \epsilon_{it}$. I only observe $y_{it}$ if $y_{it}\leq z_{it}$, otherwise $y_{it}=z_{it}$. The trick is that $z_{it}$ ...
user avatar
2 votes
1 answer
68 views

Picking a model threshold based on Validation set or Test set

I have developed a machine learning model to predict a quantitative output for medical diagnosis (low bone density). I want to convert the model output to a binary outcome and compare it to the gold-...
user avatar
  • 35
0 votes
0 answers
5 views

Why does an operational domain isn't an MV-set and why is it a problem for images and computer vision?

I am working on formalizing the operational domain of an algorithm. If the operational domain is the set $D=\{x, g(x)=1\}$ where $g$ is the selector that says when the predictor $f$ can be applied. $g(...
user avatar
0 votes
0 answers
7 views

Why does the statistical nature of the coverage of an operational domain mathematical formalisation is a problem for images and computer vision?

I am working on formalizing the operational domain of an algorithm. If the operational domain is the set $D=\{x, g(x)=1\}$ where $g$ is the selector that says when the predictor $f$ can be applied. $g(...
user avatar
2 votes
0 answers
25 views

Define outliers in correlation with right-skewed data (log-log plot)

I have a dataset of counts of occurrences of variables in different classes. For each class, I have an equivalent control created by shuffling the dataset. For instance, this could be words from ...
user avatar
  • 73
0 votes
0 answers
113 views

Given a multiclass classifier, calculate one threshold per class to maximize recall under precision constraint

Given a classifier $f$, $N$ possible classes, and an input $x$, $f$ produces a class from $[1,..., N]$ and its matching confidence $[0,...,100]$. Then I run $f$ on a large set of examples $X$, and I ...
user avatar
  • 53
1 vote
0 answers
289 views

Find the best threshold for logistic regression?

I am working on a customer purchase problem. I have 150 campaigns sent by email (or adds if you prefer), that I denote C0, C1 ......
user avatar
0 votes
0 answers
10 views

Adjusting decision thresholds for dataset with seasonality component

I am working with data on an infection which exhibits seasonality over the year - the disease tends to be more prevalent during the rainy seasons compared with dry seasons. I'm looking at a binary ...
user avatar
1 vote
1 answer
88 views

Using statistics to determine a trigger/threshold value between continuous variables

I'm working with a few continuous variables, as below: ...
user avatar
  • 11
0 votes
0 answers
45 views

classification ML model: probability of positive label knowing the model score

Question at the intersection of ML and statistics. I built a binary classification ML model, that for each input observation x outputs the probability p(x) in (0,1) that x belongs to the positive ...
user avatar
  • 111
1 vote
1 answer
33 views

XGBoost - does it make sense that accuracy decreases as threshold increases?

I'm using XGBoost for a classification problem, and if I need to check how accuracy changes as a function of threshold. As a result, I got that accuracy decreases as the threshold value increases (see ...
user avatar
  • 137
0 votes
0 answers
63 views

Confirmatory Factor Analysis factor loading cut off threshold

To conduct Confirmatory Factor Analysis (CFA), what is the usual cut off threshold for items' factor loading on their factors? I have found justifications for a more conventional .40 or a more ...
user avatar
7 votes
3 answers
431 views

Are thresholds for logistic regression models prevalence-specific?

I wonder if thresholds for logistic regression models are prevalence-specific. I assume that they are, however, I am not sure about the basic statistical principles behind it and how to deal with the ...
user avatar
  • 80
2 votes
0 answers
103 views

Multi-class optimal threshold selection

What are the methods typically used for optimal threshold selection of multiclass classification problem ? I implemented the detector/classifier and generated the confusion matrix, but would like to ...
user avatar
0 votes
0 answers
140 views

Increasing precision for one label in multiclass classification

I am doing multiclass classification for 3 labell with neural net. The model works fine but when I check precision/recall per label in validation set I can see that precision is a little bit too low ...
user avatar
0 votes
0 answers
125 views

Practical calculation of EER (equal error rate) for biometric tasks

I am recently experimenting with the speaker recognition task. So, EER is calculated for a threshold FAR = FRR. Now, my question is how can I calculate this given I ...
user avatar
1 vote
0 answers
37 views

How to set optimized thresholds to turn a two-class data into a three-class data?

Let's say I have a column, (let us call it column $I$) that has two classes, $[-1,1]$. this column is the prediction of a classification model so it has vectors of probabilities that looks like, for ...
user avatar
4 votes
1 answer
503 views

Is threshold moving unnecessary in balanced classification problem?

As far as I know, the threshold moving is needed in imbalanced classification problems. The reason why we have to adjust the decision threshold is as follows: Most machine learning algorithms are ...
user avatar
  • 501
1 vote
0 answers
15 views

Parameter tuning in classification

Recently, I am digging into the selection of tuning parameters in a binary classification. I gathered information by googling and the following is my organization. We can distinct binary ...
user avatar
  • 501
0 votes
0 answers
29 views

Decision Threshold (discriminant) threshold and accuracy in binary classification

In a binary classification problem, we usually use the ROC curve to choose the decision threshold. (the concept of the decision threshold: if an unit's probability of 1 is bigger than the threshold, ...
user avatar
  • 501
1 vote
0 answers
42 views

Order statistics and sample size

How do I estimate the odds of the highest value elements of samples from two populations, A and B, exceeding a threshold value, where the same size of A is larger than that of B? Even if A and B have ...
user avatar
  • 11
0 votes
1 answer
68 views

How to create a threshold based on a classification model?

I have a data-set that is comparing two groups (Injury only (baseline) and Surgical Modification) across multiple injury severity values (Peak Force). Reference scatter plot for context: BBB score - ...
user avatar
  • 121
0 votes
1 answer
18 views

Find treshold to separate two classes based on single predictor

I have a binary output variable (not healthy, healthy) that I want to classify. I found based on univariate analysis that one of my independent predictors already tells apart both classes perfectly. ...
user avatar
  • 164
0 votes
0 answers
54 views

Cross-validation / Threshold moving when training is balanced but test is imbalanced?

I have a binary text classification problem where texts of class 0 account for ~95% of cases and class 1 for ~5%. I put some effort until having a decently sized, balanced manually labeled subset (7k) ...
user avatar
  • 63
0 votes
0 answers
32 views

Least square for binary classification {0, 1}: is the threshold always equal to 0.5? [duplicate]

When using least square for classification of two classes labeled as 0 and 1, the value 0.5 is a common choice for the threshold. Intuitively, it makes sense, but is it always optimal? In which sense?
user avatar
  • 287
1 vote
0 answers
264 views

Finding Cosine similarity threshold (cut-off point) with Logistic Regression

I want to preprocess my data sets, removing similar articles. To do this, I'll perform cosine similarity, I don't know the best threshold (cut-off point) of my data set. I Found in this paper, there's ...
user avatar
7 votes
1 answer
106 views

Is there a name for the increase in variance upon remeasurement after subsetting with a cut-off value?

Context: My problem relates to estimating effect sizes, such as Cohen's d, when looking at a subset of the population defined by a cut-off threshold. This effect size is the difference in two ...
user avatar
2 votes
0 answers
17 views

Defining a threshold using aggregate data

I hoped for some advice - sorry if this is a very daft quesion. I have aggregate data on a medical test taken in both healthy people & patients with disease. The test provides a result on a scale ...
user avatar

1
2 3 4 5