Questions tagged [threshold]

Used (1) for discrete classification (if an instance's predicted probability exceeds a threshold, classify as TRUE, otherwise FALSE), or (2) for discretizing/binning continuous data. *If you are tempted to use this tag, PLEASE read the tag wiki!*

Filter by
Sorted by
Tagged with
0
votes
2answers
32 views

Is it ok a threshold of 0?

I am dealing with a classification problem with a dataset containing 60k rows: 69k are negative class, and 1k is positive. I trained my models and I obtained the confusion matrices with a threshold of ...
2
votes
0answers
38 views

goodness of fit for psychometric data (perceptual threshold)

I'm running an experiment on perceptual thresholds in audio. I'll try not to bog you down with too many details: The experiment is about vibrato speed; specifically, when can you tell the difference ...
0
votes
0answers
22 views

test how good a threshold is?

I have developed a threshold and wondering how can I test that threshold to see how good it is? I tried precision, recall, and F1 score metrics, and the results were promising. However, I'm wondering ...
1
vote
0answers
8 views

Classification in LDa [closed]

I am new at using linear Discriminant Functions. Currently I'm finding it difficult to find the threshold for three classes. And creating an algorithm that assigns each parameters to any of the ...
1
vote
1answer
18 views

R - Change threshold value for Random Forest classifier

After I plotted ROC and printed out the portion of values that I think it balances True Positive and False Positive. Say below. ...
0
votes
0answers
8 views

How to calculate a single threshold value to measure the mean differences are meaningful?

I have a dataset with two groups: Region A (N=3980) and Region B (N=880). The dataset is very simple which has a measurement from a sensor. The values are generally very small. I then performed Mann-...
2
votes
0answers
20 views

Why does the Youden rule does not recommend a threshold of 0.5 on balanced data?

Suppose I have a logistic regression model estimated using a balanced target (equal group sizes). My questions concern the optimal threshold for prediction and it's relationship with the Youden's rule ...
3
votes
1answer
21 views

Find the ideal threshold with the test data?

I'm working with unbalanced data (2% of the class yes and 98% of the class no). Regardless of the evaluation metric chosen in the training, I have obtained low sensitivity and high specificity. For ...
0
votes
1answer
22 views

What would be a good threshold value to determine the differences between means?

We have a dataset with 5 outcomes. The outcomes are categorical. We ran classification model and obtained the classification and prediction probabilities for each outcomes separately. For business ...
0
votes
1answer
37 views
0
votes
0answers
11 views

How to determine a binary classifier threshold for unsupervised algorithms?

I am trying to create a model that can distinguish between normal and anomalous data. The training dataset contains non-anomalous data only. I am using an autoencoder that gets trained on this data to ...
0
votes
0answers
194 views

Recreating Grid Search (k fold) Cross Validation Functions in R

I am working on a binary classification problem with an imbalanced dataset. I have decided to use a random forest model. I then trained the random forest model (grid search cross validation) using ...
2
votes
0answers
73 views

Thresholds for Bhattacharyya coefficient - when do the distributions differ significantly

The Bhattacharyya coefficient of two discrete probability distributions is defined as $$ BC(p,q) = \sum_{i=1}^n \sqrt{p_iq_i}. $$ This coefficient lies within the interval $[0,1]$ and if $p=q$ then it ...
1
vote
1answer
52 views

How to calculate the best threshold value

I'm working at a motor insurance company and want to build a business rule to detect fraud cases based on the damage value. I have a historical data set that contains a list of accidents info, damage ...
0
votes
1answer
21 views

Classification model with multiple thresholds

A model usually has one threshold, for example 0.5, if anything is greater than 0.5 we predict it as 1 vice versa. However, there are some features that impacts the output probability. Let's say ...
0
votes
0answers
41 views

What test to use for threshold

I've got a problem where, with my basic knowlegde of statistical tests, I'm not sure how to solve it. I have a dataset with data on the execution of a process. This includes: Time the process lasted ...
0
votes
0answers
14 views

statistic for defining which of two distributions a point comes from?

I have 2 samples from what I can assume are 2 different distributions, $X$ and $Y$ (I don't know anything about the true distributions, but have the empirical distributions of the samples). How can I ...
1
vote
0answers
123 views

AUC ROC and Varying Thresholds?

I understand that the ROC curve will plot the sensitivity vs FPR for varying thresholds. For my SVM ML model, I desire a good sensitivity score so I have decreased the threshold to make a positive ...
1
vote
1answer
76 views

Anomaly detection using Mahalanobis distance

I am using Mahalanobis distance to identify outliers. I am training using kind of one class classification,by training only on positive samples and trying to predict negative samples using distance ...
0
votes
0answers
39 views

P-values for sensitivity/specificity when only improvement is possible

Imagine two diagnostic tests in a single population at risk of the disease, one of which is always "more strict" than the other, meaning that if test 1 is positive, test 2 will be positive ...
4
votes
1answer
115 views

How can proper scoring rules optimize the probabilistic prediction compared to improper scoring rules?

I understand the fundamentals in the decision theory about accuracy being an improper scoring rule compared to other proper scoring rules like ...
0
votes
1answer
50 views

Set proper threshold for binary prediction in ElasticNet

I have long been struggling with setting a valid threshold t for predicting my binary logistic model and hereafter evaluate how well it performs (see code below). I ...
2
votes
0answers
55 views

Poisson Counting Process over Variable Threshold

Let's consider a Poisson Counting Process $N(t)$ with generation rate $\lambda$ and a piece-wise constant threshold $M(t)$ that changes with time. $$ M(t)=m_i \ \ for \ \ t\in(t_{i-1},t_{i}],\ i\in \...
2
votes
1answer
157 views

Is my AUC too good to be true?

I've been going in circles for months...I want to generate a list of thresholds from my training data so I can see all the thresholds at every sensitivity/1-specificity of a model. You can do this ...
4
votes
0answers
63 views

Does distance from the decision boundary suggest higher confidence that the class prediction is correct using SVM?

Does further distance from the decision boundary threshold suggest higher confidence that the class prediction is correct when using SVM with probability estimates enabled? This is not a question ...
1
vote
1answer
551 views

How to set a threshold on softmax probabilities in a multi-class classification task?

I have a large image dataset that was classified by a ConvNet into different classes (objects). For each image the top-1 softmax probability is given, ranging between 0 and 1. It´s the output of a ...
1
vote
1answer
43 views

Does a threshold effect the training or testing fold in cross validation?

I am trying to better understand how changing a threshold affects a cross validation model. So if you trained a random forest model, the default threshold is ...
1
vote
1answer
227 views

threshold choice for binary classifier: on training, validation or test set?

I have a binary classification problem where I perform cross validation on the training set (currently 80% of the examples) and then evaluate results on a test set. I use cross validation for finding ...
0
votes
0answers
35 views

Is a threshold for categorical variables necessary in GLM?

I was wondering if there is a specific guideline or rule of thumb somewhere regarding the use of categorical predictor levels with low representation in the dataset? I have a few categorical ...
0
votes
0answers
17 views

Risk score stratification with k-means

I have created a risk score (point-based risk score) to predict time to brain metastases from covariates selected with a backward approach. My risk score ranges from 0 to 45. I need to classify my ...
0
votes
0answers
39 views

One-Class SVM threshold parameter

I have to implement metrics FPR at 95%TPR. In order to do that I have to look for the different decisions of OCSVM dependent on the threshold. If I execute this simple code: ...
0
votes
0answers
23 views

How to calculate the threshold value for a variable

In a question-answer platform, in each week the platform releases a new question. In total there are 15 questions. There are these three types of users. Log in every week, and answer in the correct ...
0
votes
0answers
17 views

Very basic question about predictability

I have two set of options and have data from each showing potential outcomes (distances). I want to determine which option is more likely to exceed a target value, ie cross a threshold - the amount by ...
0
votes
1answer
203 views

Threshold for LDA in the case of three classes

The result of discriminant analysis for the three classes are two discriminant functions LD1 and LD2. In the case of two classes, there is one discriminant function LD1, and the threshold of ...
0
votes
0answers
43 views

Why does Precision always equal Recall at Threshold 0.5?

I ran a few models through Google AutoML Natural Language Single-Label Classification. When I get to the evaluation page, I always see that the precision and recall are equal at a threshold of 0.5. ...
1
vote
1answer
63 views

Normalize to sum to unity, but assign minimum value

I want to normalize some sample that I have to sum to unity: $$ W = \sum_{i=1}^{n} w_i $$ $$ normalized_i = w_i/W $$ However, it might happen that the values of ...
0
votes
0answers
19 views

Stacking model: is finding best threshold necessary?

After performing a stacking model (with rstudio), is it necessary to choose the best threshold for it? In general after finding the best model among all the fitted models , you have to choose the ...
1
vote
0answers
32 views

Optimal threshold on classification probability [closed]

I have a question regarding threshold on classification. In the problem listed below, are they stating that λ01 and λ10 are equal to zero? Listed below is the question. Consider a case where we have ...
1
vote
1answer
127 views

How to modify Presicion and recall of GLM (Logit) in R? [closed]

I fit a logistic regression model with an unbalanced population in R. The problem that I am getting is I have 0.4 for precision and 0.0018 for recall, so I want to modify the threshold in order to ...
0
votes
0answers
21 views

Feature subset selection or find classifier threshold first for logistic regression?

I have a baseline model with 137 variables and 300 observations, and I am trying to solve a classification problem with a logistic model. I understand that the base model won't have much value. In ...
0
votes
0answers
15 views

Materiality Threshold - statistics newbie (help!)

I am not sure how to approach the following situation and am looking for advice on how to develop a quantitative measure - The situation: I am comparing a rate calculated from two different sources, ...
3
votes
1answer
89 views

My machine learning models have reproducible results generally, but not at specific classification thresholds. What does this mean?

My binary classifier machine learning pipelines are producing generalizable results in that they are predicting the positive class (probability > .5) with the same precision on unseen data in ...
2
votes
2answers
106 views

Probability of Normal (Gaussian) random walk crossing threshold within k steps

Let $x[n]$ be a Gaussian random walk, so $x[0] = 0$ and $x[n+1] = x[n] + v$, where $v$ is an independent random variable with normal distribution, $0$ mean and standard deviation $s$. What is the ...
1
vote
1answer
41 views

ROC curves: can a cut-point (cut-off) be “useful”, or is it a term reserved to parameters only?

This is a semantic question: When analyzing Receiver Operating Characteric (ROC) curves, we talk about the "usefulness" of a parameter based on its Area Under the Curve (AUC). "Useful" here refers to ...
9
votes
1answer
218 views

Decision rule as a hyper-parameter in LASSO

I have a question that is related to the following: Is decision threshold a hyperparameter in logistic regression? but would like some clarification. The general consensus is that the decision rule ...
1
vote
1answer
49 views

Determining break point/threshold in species richness in relation to land cover from multi-species occupancy models in R

I am conducting research in which I would like to see if I can detect a threshold response in mammal species richness in relation to the surrounding level of forest across study sites. I plan to use ...
1
vote
1answer
72 views

Bayesian liability threshold model

Let $\bf{y}$ denote a vector of binary data, such as whether a group of individuals suffer from a particular disease, and let $\bf{X}$ denote a matrix of potential predictors, including an intercept ...
2
votes
0answers
11 views

Best methods to integrate summed probability and uncertainty from multiple sources?

I am trying to calculate risk of coastal flooding. In our model, we have 4 major sources of uncertainty: Elevation at location (height + gaussian error) Sea level rise (value at any given year + ...
0
votes
1answer
336 views

How to decide threshold values for AUC?

With respect to the pROC package for R(https://rdrr.io/cran/pROC/man/ggroc.html). ...
2
votes
0answers
50 views

Threshold crossing: Integrating posterior distribution over a forecast period

Suppose I have some time series $\{Y_t\}_{t=1}^T$ and I have come up with some probabilistic model: $$p(Y_{t+h} \mid Y_{1:t}, \theta)$$ for any horizon $h$, and some parameters $\theta$. Now ...