Questions tagged [threshold]

Used (1) for discrete classification (if an instance's predicted probability exceeds a threshold, classify as TRUE, otherwise FALSE), or (2) for discretizing/binning continuous data. *If you are tempted to use this tag, PLEASE read the tag wiki!*

Filter by
Sorted by
Tagged with
1
vote
0answers
229 views

Find the best threshold for logistic regression?

I am working on a customer purchase problem. I have 150 campaigns sent by email (or adds if you prefer), that I denote C0, C1 ......
0
votes
0answers
9 views

Adjusting decision thresholds for dataset with seasonality component

I am working with data on an infection which exhibits seasonality over the year - the disease tends to be more prevalent during the rainy seasons compared with dry seasons. I'm looking at a binary ...
1
vote
1answer
30 views

Using statistics to determine a trigger/threshold value between continuous variables

I'm working with a few continuous variables, as below: ...
0
votes
0answers
27 views

classification ML model: probability of positive label knowing the model score

Question at the intersection of ML and statistics. I built a binary classification ML model, that for each input observation x outputs the probability p(x) in (0,1) that x belongs to the positive ...
0
votes
1answer
13 views

XGBoost - does it make sense that accuracy decreases as threshold increases?

I'm using XGBoost for a classification problem, and if I need to check how accuracy changes as a function of threshold. As a result, I got that accuracy decreases as the threshold value increases (see ...
0
votes
0answers
9 views

Confirmatory Factor Analysis factor loading cut off threshold

To conduct Confirmatory Factor Analysis (CFA), what is the usual cut off threshold for items' factor loading on their factors? I have found justifications for a more conventional .40 or a more ...
8
votes
3answers
316 views

Are thresholds for logistic regression models prevalence-specific?

I wonder if thresholds for logistic regression models are prevalence-specific. I assume that they are, however, I am not sure about the basic statistical principles behind it and how to deal with the ...
2
votes
0answers
29 views

Multi-class optimal threshold selection

What are the methods typically used for optimal threshold selection of multiclass classification problem ? I implemented the detector/classifier and generated the confusion matrix, but would like to ...
0
votes
0answers
23 views

Increasing precision for one label in multiclass classification

I am doing multiclass classification for 3 labell with neural net. The model works fine but when I check precision/recall per label in validation set I can see that precision is a little bit too low ...
0
votes
0answers
7 views

Practical calculation of EER (equal error rate) for biometric tasks

I am recently experimenting with the speaker recognition task. So, EER is calculated for a threshold FAR = FRR. Now, my question is how can I calculate this given I ...
1
vote
0answers
26 views

How to set optimized thresholds to turn a two-class data into a three-class data?

Let's say I have a column, (let us call it column $I$) that has two classes, $[-1,1]$. this column is the prediction of a classification model so it has vectors of probabilities that looks like, for ...
2
votes
1answer
100 views

Is threshold moving unnecessary in balanced classification problem?

As far as I know, the threshold moving is needed in imbalanced classification problems. The reason why we have to adjust the decision threshold is as follows: Most machine learning algorithms are ...
1
vote
0answers
13 views

Parameter tuning in classification

Recently, I am digging into the selection of tuning parameters in a binary classification. I gathered information by googling and the following is my organization. We can distinct binary ...
0
votes
0answers
12 views

Decision Threshold (discriminant) threshold and accuracy in binary classification

In a binary classification problem, we usually use the ROC curve to choose the decision threshold. (the concept of the decision threshold: if an unit's probability of 1 is bigger than the threshold, ...
1
vote
0answers
39 views

Order statistics and sample size

How do I estimate the odds of the highest value elements of samples from two populations, A and B, exceeding a threshold value, where the same size of A is larger than that of B? Even if A and B have ...
1
vote
1answer
60 views

How to create a threshold based on a classification model?

I have a data-set that is comparing two groups (Injury only (baseline) and Surgical Modification) across multiple injury severity values (Peak Force). Reference scatter plot for context: BBB score - ...
0
votes
0answers
15 views

How to calculate statistical threshold/cutoff point given multiple variables across two groups?

I have a data-set that is comparing two groups (Baseline and Surgical Modification) across multiple injury severity (Peak Force). Reference scatter plot for context: BBB score - yaxis is what is ...
0
votes
1answer
15 views

Find treshold to separate two classes based on single predictor

I have a binary output variable (not healthy, healthy) that I want to classify. I found based on univariate analysis that one of my independent predictors already tells apart both classes perfectly. ...
0
votes
0answers
24 views

Cross-validation / Threshold moving when training is balanced but test is imbalanced?

I have a binary text classification problem where texts of class 0 account for ~95% of cases and class 1 for ~5%. I put some effort until having a decently sized, balanced manually labeled subset (7k) ...
0
votes
0answers
29 views

Least square for binary classification {0, 1}: is the threshold always equal to 0.5? [duplicate]

When using least square for classification of two classes labeled as 0 and 1, the value 0.5 is a common choice for the threshold. Intuitively, it makes sense, but is it always optimal? In which sense?
0
votes
0answers
79 views

Finding Cosine similarity threshold (cut-off point) with Logistic Regression

I want to preprocess my data sets, removing similar articles. To do this, I'll perform cosine similarity, I don't know the best threshold (cut-off point) of my data set. I Found in this paper, there's ...
7
votes
1answer
98 views

Is there a name for the increase in variance upon remeasurement after subsetting with a cut-off value?

Context: My problem relates to estimating effect sizes, such as Cohen's d, when looking at a subset of the population defined by a cut-off threshold. This effect size is the difference in two ...
2
votes
0answers
13 views

Defining a threshold using aggregate data

I hoped for some advice - sorry if this is a very daft quesion. I have aggregate data on a medical test taken in both healthy people & patients with disease. The test provides a result on a scale ...
0
votes
0answers
12 views

Ordinal regression on SPSS

I'm doing an ordinal regression on SPSS and have one question: what is the meaning of the THRESHOLD in the Parameter Estimates? Most of the resources I found just say that those are not interesting to ...
2
votes
2answers
81 views

How would you find a p threshold for a binary classification prediction?

Lets say that there's a binary classification problem where $X$ ∈ $R_p$ and $Y ∈ \{0,1\} $ and $Pr(Y = 1 | X = x) = p$ for $p$ in $[0,1]$. There is a loss function $L_{falseneg} > 0$ for false ...
0
votes
0answers
11 views

Adaptive cutpoint selection on ROC curves based on changing environments

I have built a classic binary classifier and constructed a ROC curve for it, like the following: In this case, the positive class represents "bad" things that should be excluded. In the ...
0
votes
0answers
48 views

Selecting Binary Classification Probability Threshold [duplicate]

I have a binary classification problem I have modeled and I'm trying to determine the best way to select my probability threshold. Here was my modeling approach: Create a training and testing set. My ...
0
votes
0answers
22 views

How to decide where to cut off tailing

Background We have a model that calculates transport of substances through the air to calculate deposition from source emissions. However, it is modeled up to a large distance from the source until ...
0
votes
0answers
18 views

ROC curve with multiple thresholds / equivalent rank measure

I have a model that has X features, and I can create ROC curves based on the residuals for each feature (so that each feature has its own curve, that determines how good that feature's residuals are ...
0
votes
0answers
18 views

ROC convex hull - realisable classifiers

In the paper Realisable Classifiers Theorem 1 shows that there exists a realisable classifier r_i which lies on a line L_ab ...
0
votes
0answers
19 views

How do I find a threshold to minimise some values? Relevant graph included

I am trying to solve two sub problems based on this graph. My overarching problem is to predict the coordinates of a point, and give a confidence to the prediction. The y-axis is kind of arbitrary, ...
0
votes
2answers
85 views

Is it ok a threshold of 0?

I am dealing with a classification problem with a dataset containing 60k rows: 69k are negative class, and 1k is positive. I trained my models and I obtained the confusion matrices with a threshold of ...
2
votes
0answers
45 views

goodness of fit for psychometric data (perceptual threshold)

I'm running an experiment on perceptual thresholds in audio. I'll try not to bog you down with too many details: The experiment is about vibrato speed; specifically, when can you tell the difference ...
0
votes
0answers
28 views

test how good a threshold is?

I have developed a threshold and wondering how can I test that threshold to see how good it is? I tried precision, recall, and F1 score metrics, and the results were promising. However, I'm wondering ...
1
vote
0answers
15 views

Classification in LDa [closed]

I am new at using linear Discriminant Functions. Currently I'm finding it difficult to find the threshold for three classes. And creating an algorithm that assigns each parameters to any of the ...
1
vote
1answer
44 views

R - Change threshold value for Random Forest classifier

After I plotted ROC and printed out the portion of values that I think it balances True Positive and False Positive. Say below. ...
0
votes
0answers
14 views

How to calculate a single threshold value to measure the mean differences are meaningful?

I have a dataset with two groups: Region A (N=3980) and Region B (N=880). The dataset is very simple which has a measurement from a sensor. The values are generally very small. I then performed Mann-...
2
votes
0answers
25 views

Why does the Youden rule does not recommend a threshold of 0.5 on balanced data?

Suppose I have a logistic regression model estimated using a balanced target (equal group sizes). My questions concern the optimal threshold for prediction and it's relationship with the Youden's rule ...
3
votes
1answer
77 views

Find the ideal threshold with the test data?

I'm working with unbalanced data (2% of the class yes and 98% of the class no). Regardless of the evaluation metric chosen in the training, I have obtained low sensitivity and high specificity. For ...
0
votes
1answer
36 views

What would be a good threshold value to determine the differences between means?

We have a dataset with 5 outcomes. The outcomes are categorical. We ran classification model and obtained the classification and prediction probabilities for each outcomes separately. For business ...
0
votes
1answer
39 views
0
votes
0answers
38 views

How to determine a binary classifier threshold for unsupervised algorithms?

I am trying to create a model that can distinguish between normal and anomalous data. The training dataset contains non-anomalous data only. I am using an autoencoder that gets trained on this data to ...
0
votes
0answers
334 views

Recreating Grid Search (k fold) Cross Validation Functions in R

I am working on a binary classification problem with an imbalanced dataset. I have decided to use a random forest model. I then trained the random forest model (grid search cross validation) using ...
2
votes
0answers
143 views

Thresholds for Bhattacharyya coefficient - when do the distributions differ significantly

The Bhattacharyya coefficient of two discrete probability distributions is defined as $$ BC(p,q) = \sum_{i=1}^n \sqrt{p_iq_i}. $$ This coefficient lies within the interval $[0,1]$ and if $p=q$ then it ...
1
vote
2answers
187 views

How to calculate the best threshold value

I'm working at a motor insurance company and want to build a business rule to detect fraud cases based on the damage value. I have a historical data set that contains a list of accidents info, damage ...
1
vote
1answer
76 views

Classification model with multiple thresholds

A model usually has one threshold, for example 0.5, if anything is greater than 0.5 we predict it as 1 vice versa. However, there are some features that impacts the output probability. Let's say ...
0
votes
0answers
42 views

What test to use for threshold

I've got a problem where, with my basic knowlegde of statistical tests, I'm not sure how to solve it. I have a dataset with data on the execution of a process. This includes: Time the process lasted ...
0
votes
0answers
19 views

statistic for defining which of two distributions a point comes from?

I have 2 samples from what I can assume are 2 different distributions, $X$ and $Y$ (I don't know anything about the true distributions, but have the empirical distributions of the samples). How can I ...
1
vote
0answers
380 views

AUC ROC and Varying Thresholds?

I understand that the ROC curve will plot the sensitivity vs FPR for varying thresholds. For my SVM ML model, I desire a good sensitivity score so I have decreased the threshold to make a positive ...
1
vote
1answer
259 views

Anomaly detection using Mahalanobis distance

I am using Mahalanobis distance to identify outliers. I am training using kind of one class classification,by training only on positive samples and trying to predict negative samples using distance ...

1
2 3 4 5