Questions tagged [threshold]

Used (1) for discrete classification (if an instance's predicted probability exceeds a threshold, classify as TRUE, otherwise FALSE), or (2) for discretizing/binning continuous data. *If you are tempted to use this tag, PLEASE read the tag wiki!*

Filter by
Sorted by
Tagged with
0
votes
1answer
14 views

Find treshold to separate two classes based on single predictor

I have a binary output variable (not healthy, healthy) that I want to classify. I found based on univariate analysis that one of my independent predictors already tells apart both classes perfectly. ...
0
votes
0answers
18 views

Cross-validation / Threshold moving when training is balanced but test is imbalanced?

I have a binary text classification problem where texts of class 0 account for ~95% of cases and class 1 for ~5%. I put some effort until having a decently sized, balanced manually labeled subset (7k) ...
0
votes
0answers
14 views

How to calculate a threshold?

I have a dataset with 2 features. One feature is process and the other one is the coefficient of variation value of the process. Now I need to find a threshold value, something like "anything ...
0
votes
0answers
27 views

Least square for binary classification {0, 1}: is the threshold always equal to 0.5? [duplicate]

When using least square for classification of two classes labeled as 0 and 1, the value 0.5 is a common choice for the threshold. Intuitively, it makes sense, but is it always optimal? In which sense?
0
votes
0answers
28 views

Finding Cosine similarity threshold (cut-off point) with Logistic Regression

I want to preprocess my data sets, removing similar articles. To do this, I'll perform cosine similarity, I don't know the best threshold (cut-off point) of my data set. I Found in this paper, there's ...
5
votes
1answer
86 views

Is there a name for the increase in variance upon remeasurement after subsetting with a cut-off value?

Context: My problem relates to estimating effect sizes, such as Cohen's d, when looking at a subset of the population defined by a cut-off threshold. This effect size is the difference in two ...
2
votes
0answers
12 views

Defining a threshold using aggregate data

I hoped for some advice - sorry if this is a very daft quesion. I have aggregate data on a medical test taken in both healthy people & patients with disease. The test provides a result on a scale ...
0
votes
0answers
11 views

Ordinal regression on SPSS

I'm doing an ordinal regression on SPSS and have one question: what is the meaning of the THRESHOLD in the Parameter Estimates? Most of the resources I found just say that those are not interesting to ...
0
votes
0answers
14 views

How would you find a p threshold for a binary classification prediction?

Lets say that there's a binary classification problem where $X$ ∈ $R_p$ and $Y ∈ \{0,1\} $ and $Pr(Y = 1 | X = x) = p$ for $p$ in $[0,1]$. There is a loss function $L_{falseneg} > 0$ for false ...
0
votes
0answers
11 views

Adaptive cutpoint selection on ROC curves based on changing environments

I have built a classic binary classifier and constructed a ROC curve for it, like the following: In this case, the positive class represents "bad" things that should be excluded. In the ...
0
votes
0answers
8 views

Selecting Binary Classification Probability Threshold [duplicate]

I have a binary classification problem I have modeled and I'm trying to determine the best way to select my probability threshold. Here was my modeling approach: Create a training and testing set. My ...
0
votes
0answers
20 views

How to decide where to cut off tailing

Background We have a model that calculates transport of substances through the air to calculate deposition from source emissions. However, it is modeled up to a large distance from the source until ...
0
votes
0answers
10 views

ROC curve with multiple thresholds / equivalent rank measure

I have a model that has X features, and I can create ROC curves based on the residuals for each feature (so that each feature has its own curve, that determines how good that feature's residuals are ...
0
votes
0answers
11 views

ROC convex hull - realisable classifiers

In the paper Realisable Classifiers Theorem 1 shows that there exists a realisable classifier r_i which lies on a line L_ab ...
0
votes
0answers
19 views

Decision threshold for classification together with RandomizedSearchCV

I was wondering what the correct workflow would be to obtain the best threshold of the classifier (when we want to minimize both FN and FP) but also want to do hyperparameter optimisation. I thought ...
0
votes
0answers
18 views

How do I find a threshold to minimise some values? Relevant graph included

I am trying to solve two sub problems based on this graph. My overarching problem is to predict the coordinates of a point, and give a confidence to the prediction. The y-axis is kind of arbitrary, ...
0
votes
2answers
45 views

Is it ok a threshold of 0?

I am dealing with a classification problem with a dataset containing 60k rows: 69k are negative class, and 1k is positive. I trained my models and I obtained the confusion matrices with a threshold of ...
2
votes
0answers
38 views

goodness of fit for psychometric data (perceptual threshold)

I'm running an experiment on perceptual thresholds in audio. I'll try not to bog you down with too many details: The experiment is about vibrato speed; specifically, when can you tell the difference ...
0
votes
0answers
26 views

test how good a threshold is?

I have developed a threshold and wondering how can I test that threshold to see how good it is? I tried precision, recall, and F1 score metrics, and the results were promising. However, I'm wondering ...
1
vote
0answers
10 views

Classification in LDa [closed]

I am new at using linear Discriminant Functions. Currently I'm finding it difficult to find the threshold for three classes. And creating an algorithm that assigns each parameters to any of the ...
1
vote
1answer
21 views

R - Change threshold value for Random Forest classifier

After I plotted ROC and printed out the portion of values that I think it balances True Positive and False Positive. Say below. ...
0
votes
0answers
9 views

How to calculate a single threshold value to measure the mean differences are meaningful?

I have a dataset with two groups: Region A (N=3980) and Region B (N=880). The dataset is very simple which has a measurement from a sensor. The values are generally very small. I then performed Mann-...
2
votes
0answers
21 views

Why does the Youden rule does not recommend a threshold of 0.5 on balanced data?

Suppose I have a logistic regression model estimated using a balanced target (equal group sizes). My questions concern the optimal threshold for prediction and it's relationship with the Youden's rule ...
3
votes
1answer
22 views

Find the ideal threshold with the test data?

I'm working with unbalanced data (2% of the class yes and 98% of the class no). Regardless of the evaluation metric chosen in the training, I have obtained low sensitivity and high specificity. For ...
0
votes
1answer
25 views

What would be a good threshold value to determine the differences between means?

We have a dataset with 5 outcomes. The outcomes are categorical. We ran classification model and obtained the classification and prediction probabilities for each outcomes separately. For business ...
0
votes
1answer
38 views
0
votes
0answers
22 views

How to determine a binary classifier threshold for unsupervised algorithms?

I am trying to create a model that can distinguish between normal and anomalous data. The training dataset contains non-anomalous data only. I am using an autoencoder that gets trained on this data to ...
0
votes
0answers
247 views

Recreating Grid Search (k fold) Cross Validation Functions in R

I am working on a binary classification problem with an imbalanced dataset. I have decided to use a random forest model. I then trained the random forest model (grid search cross validation) using ...
2
votes
0answers
89 views

Thresholds for Bhattacharyya coefficient - when do the distributions differ significantly

The Bhattacharyya coefficient of two discrete probability distributions is defined as $$ BC(p,q) = \sum_{i=1}^n \sqrt{p_iq_i}. $$ This coefficient lies within the interval $[0,1]$ and if $p=q$ then it ...
1
vote
1answer
69 views

How to calculate the best threshold value

I'm working at a motor insurance company and want to build a business rule to detect fraud cases based on the damage value. I have a historical data set that contains a list of accidents info, damage ...
0
votes
1answer
32 views

Classification model with multiple thresholds

A model usually has one threshold, for example 0.5, if anything is greater than 0.5 we predict it as 1 vice versa. However, there are some features that impacts the output probability. Let's say ...
0
votes
0answers
41 views

What test to use for threshold

I've got a problem where, with my basic knowlegde of statistical tests, I'm not sure how to solve it. I have a dataset with data on the execution of a process. This includes: Time the process lasted ...
0
votes
0answers
17 views

statistic for defining which of two distributions a point comes from?

I have 2 samples from what I can assume are 2 different distributions, $X$ and $Y$ (I don't know anything about the true distributions, but have the empirical distributions of the samples). How can I ...
1
vote
0answers
226 views

AUC ROC and Varying Thresholds?

I understand that the ROC curve will plot the sensitivity vs FPR for varying thresholds. For my SVM ML model, I desire a good sensitivity score so I have decreased the threshold to make a positive ...
1
vote
1answer
143 views

Anomaly detection using Mahalanobis distance

I am using Mahalanobis distance to identify outliers. I am training using kind of one class classification,by training only on positive samples and trying to predict negative samples using distance ...
0
votes
0answers
42 views

P-values for sensitivity/specificity when only improvement is possible

Imagine two diagnostic tests in a single population at risk of the disease, one of which is always "more strict" than the other, meaning that if test 1 is positive, test 2 will be positive ...
4
votes
1answer
146 views

How can proper scoring rules optimize the probabilistic prediction compared to improper scoring rules?

I understand the fundamentals in the decision theory about accuracy being an improper scoring rule compared to other proper scoring rules like ...
1
vote
1answer
78 views

Set proper threshold for binary prediction in ElasticNet

I have long been struggling with setting a valid threshold t for predicting my binary logistic model and hereafter evaluate how well it performs (see code below). I ...
2
votes
0answers
57 views

Poisson Counting Process over Variable Threshold

Let's consider a Poisson Counting Process $N(t)$ with generation rate $\lambda$ and a piece-wise constant threshold $M(t)$ that changes with time. $$ M(t)=m_i \ \ for \ \ t\in(t_{i-1},t_{i}],\ i\in \...
2
votes
1answer
194 views

Is my AUC too good to be true?

I've been going in circles for months...I want to generate a list of thresholds from my training data so I can see all the thresholds at every sensitivity/1-specificity of a model. You can do this ...
4
votes
0answers
73 views

Does distance from the decision boundary suggest higher confidence that the class prediction is correct using SVM?

Does further distance from the decision boundary threshold suggest higher confidence that the class prediction is correct when using SVM with probability estimates enabled? This is not a question ...
1
vote
1answer
897 views

How to set a threshold on softmax probabilities in a multi-class classification task?

I have a large image dataset that was classified by a ConvNet into different classes (objects). For each image the top-1 softmax probability is given, ranging between 0 and 1. It´s the output of a ...
1
vote
1answer
71 views

Does a threshold effect the training or testing fold in cross validation?

I am trying to better understand how changing a threshold affects a cross validation model. So if you trained a random forest model, the default threshold is ...
1
vote
1answer
362 views

threshold choice for binary classifier: on training, validation or test set?

I have a binary classification problem where I perform cross validation on the training set (currently 80% of the examples) and then evaluate results on a test set. I use cross validation for finding ...
0
votes
0answers
49 views

Is a threshold for categorical variables necessary in GLM?

I was wondering if there is a specific guideline or rule of thumb somewhere regarding the use of categorical predictor levels with low representation in the dataset? I have a few categorical ...
0
votes
0answers
21 views

Risk score stratification with k-means

I have created a risk score (point-based risk score) to predict time to brain metastases from covariates selected with a backward approach. My risk score ranges from 0 to 45. I need to classify my ...
0
votes
0answers
51 views

One-Class SVM threshold parameter

I have to implement metrics FPR at 95%TPR. In order to do that I have to look for the different decisions of OCSVM dependent on the threshold. If I execute this simple code: ...
0
votes
0answers
28 views

How to calculate the threshold value for a variable

In a question-answer platform, in each week the platform releases a new question. In total there are 15 questions. There are these three types of users. Log in every week, and answer in the correct ...
0
votes
0answers
17 views

Very basic question about predictability

I have two set of options and have data from each showing potential outcomes (distances). I want to determine which option is more likely to exceed a target value, ie cross a threshold - the amount by ...
0
votes
1answer
308 views

Threshold for LDA in the case of three classes

The result of discriminant analysis for the three classes are two discriminant functions LD1 and LD2. In the case of two classes, there is one discriminant function LD1, and the threshold of ...

1
2 3 4 5