Questions tagged [threshold]

Used (1) for discrete classification (if an instance's predicted probability exceeds a threshold, classify as TRUE, otherwise FALSE), or (2) for discretizing/binning continuous data. *If you are tempted to use this tag, PLEASE read the tag wiki!*

Filter by
Sorted by
Tagged with
1
vote
0answers
28 views

AUC ROC and Varying Thresholds?

I understand that the ROC curve will plot the sensitivity vs FPR for varying thresholds. For my SVM ML model, I desire a good sensitivity score so I have decreased the threshold to make a positive ...
1
vote
1answer
31 views

Anomaly detection using Mahalanobis distance

I am using Mahalanobis distance to identify outliers. I am training using kind of one class classification,by training only on positive samples and trying to predict negative samples using distance ...
0
votes
0answers
29 views

P-values for sensitivity/specificity when only improvement is possible

Imagine two diagnostic tests in a single population at risk of the disease, one of which is always "more strict" than the other, meaning that if test 1 is positive, test 2 will be positive ...
4
votes
1answer
52 views

How can proper scoring rules optimize the probabilistic prediction compared to improper scoring rules?

I understand the fundamentals in the decision theory about accuracy being an improper scoring rule compared to other proper scoring rules like ...
0
votes
1answer
43 views

Set proper threshold for binary prediction in ElasticNet

I have long been struggling with setting a valid threshold t for predicting my binary logistic model and hereafter evaluate how well it performs (see code below). I ...
1
vote
0answers
45 views

Poisson Counting Process over Variable Threshold

Let's consider a Poisson Counting Process $N(t)$ with generation rate $\lambda$ and a piece-wise constant threshold $M(t)$ that changes with time. $$ M(t)=m_i \ \ for \ \ t\in(t_{i-1},t_{i}],\ i\in \...
2
votes
1answer
99 views

Is my AUC too good to be true?

I've been going in circles for months...I want to generate a list of thresholds from my training data so I can see all the thresholds at every sensitivity/1-specificity of a model. You can do this ...
1
vote
1answer
82 views

How to set a threshold on softmax probabilities in a multi-class classification task?

I have a large image dataset that was classified by a ConvNet into different classes (objects). For each image the top-1 softmax probability is given, ranging between 0 and 1. It´s the output of a ...
1
vote
1answer
28 views

Does a threshold effect the training or testing fold in cross validation?

I am trying to better understand how changing a threshold affects a cross validation model. So if you trained a random forest model, the default threshold is ...
1
vote
1answer
67 views

threshold choice for binary classifier: on training, validation or test set?

I have a binary classification problem where I perform cross validation on the training set (currently 80% of the examples) and then evaluate results on a test set. I use cross validation for finding ...
0
votes
0answers
20 views

Is a threshold for categorical variables necessary in GLM?

I was wondering if there is a specific guideline or rule of thumb somewhere regarding the use of categorical predictor levels with low representation in the dataset? I have a few categorical ...
0
votes
0answers
10 views

Risk score stratification with k-means

I have created a risk score (point-based risk score) to predict time to brain metastases from covariates selected with a backward approach. My risk score ranges from 0 to 45. I need to classify my ...
0
votes
0answers
27 views

One-Class SVM threshold parameter

I have to implement metrics FPR at 95%TPR. In order to do that I have to look for the different decisions of OCSVM dependent on the threshold. If I execute this simple code: ...
0
votes
0answers
18 views

How to calculate the threshold value for a variable

In a question-answer platform, in each week the platform releases a new question. In total there are 15 questions. There are these three types of users. Log in every week, and answer in the correct ...
0
votes
0answers
17 views

Very basic question about predictability

I have two set of options and have data from each showing potential outcomes (distances). I want to determine which option is more likely to exceed a target value, ie cross a threshold - the amount by ...
0
votes
1answer
94 views

Threshold for LDA in the case of three classes

The result of discriminant analysis for the three classes are two discriminant functions LD1 and LD2. In the case of two classes, there is one discriminant function LD1, and the threshold of ...
0
votes
0answers
19 views

Why does Precision always equal Recall at Threshold 0.5?

I ran a few models through Google AutoML Natural Language Single-Label Classification. When I get to the evaluation page, I always see that the precision and recall are equal at a threshold of 0.5. ...
1
vote
1answer
53 views

Normalize to sum to unity, but assign minimum value

I want to normalize some sample that I have to sum to unity: $$ W = \sum_{i=1}^{n} w_i $$ $$ normalized_i = w_i/W $$ However, it might happen that the values of ...
0
votes
0answers
14 views

Stacking model: is finding best threshold necessary?

After performing a stacking model (with rstudio), is it necessary to choose the best threshold for it? In general after finding the best model among all the fitted models , you have to choose the ...
1
vote
0answers
28 views

Optimal threshold on classification probability [closed]

I have a question regarding threshold on classification. In the problem listed below, are they stating that λ01 and λ10 are equal to zero? Listed below is the question. Consider a case where we have ...
1
vote
1answer
60 views

How to modify Presicion and recall of GLM (Logit) in R? [closed]

I fit a logistic regression model with an unbalanced population in R. The problem that I am getting is I have 0.4 for precision and 0.0018 for recall, so I want to modify the threshold in order to ...
0
votes
0answers
19 views

Feature subset selection or find classifier threshold first for logistic regression?

I have a baseline model with 137 variables and 300 observations, and I am trying to solve a classification problem with a logistic model. I understand that the base model won't have much value. In ...
0
votes
0answers
15 views

Materiality Threshold - statistics newbie (help!)

I am not sure how to approach the following situation and am looking for advice on how to develop a quantitative measure - The situation: I am comparing a rate calculated from two different sources, ...
3
votes
1answer
69 views

My machine learning models have reproducible results generally, but not at specific classification thresholds. What does this mean?

My binary classifier machine learning pipelines are producing generalizable results in that they are predicting the positive class (probability > .5) with the same precision on unseen data in ...
2
votes
2answers
76 views

Probability of Normal (Gaussian) random walk crossing threshold within k steps

Let $x[n]$ be a Gaussian random walk, so $x[0] = 0$ and $x[n+1] = x[n] + v$, where $v$ is an independent random variable with normal distribution, $0$ mean and standard deviation $s$. What is the ...
1
vote
1answer
39 views

ROC curves: can a cut-point (cut-off) be “useful”, or is it a term reserved to parameters only?

This is a semantic question: When analyzing Receiver Operating Characteric (ROC) curves, we talk about the "usefulness" of a parameter based on its Area Under the Curve (AUC). "Useful" here refers to ...
0
votes
0answers
11 views

Choosing Prior Distribution for AR coefficients in TAR/SETAR model

I am trying to choose the prior distribution for the AR coefficients for a general TAR model. I am tone between a truncated normal and uniform priors. Are there books or resources you can suggest for ...
9
votes
1answer
158 views

Decision rule as a hyper-parameter in LASSO

I have a question that is related to the following: Is decision threshold a hyperparameter in logistic regression? but would like some clarification. The general consensus is that the decision rule ...
0
votes
0answers
15 views

Nested Cross Validation: Choosing a Classification probability threshold

With nested CV: Inner loop for model selection, outer loop for performance evaluation. At what level can we optimize a threshold probability (vs. 0.5) to maximize sensitivity or specificity of the ...
0
votes
0answers
206 views

Tuning threshold from multiclass ROC for Gradient Boosting Classifier?

I have created a ROC curve based on the output of a multiclass Gradient Boosting Classifier (See Figure below implemented from Yellowbrick ROCAUC: https://www.scikit-yb.org/en/latest/api/classifier/...
0
votes
0answers
29 views

How do I perform data cleaning for threshold detection using method of limits?

I have a dataset collected using the Method of Limits (https://en.wikipedia.org/wiki/Psychophysics#Method_of_limits) and am looking for any approaches to outlier detection that are accepted practice. ...
1
vote
1answer
31 views

Determining break point/threshold in species richness in relation to land cover from multi-species occupancy models in R

I am conducting research in which I would like to see if I can detect a threshold response in mammal species richness in relation to the surrounding level of forest across study sites. I plan to use ...
1
vote
1answer
56 views

Bayesian liability threshold model

Let $\bf{y}$ denote a vector of binary data, such as whether a group of individuals suffer from a particular disease, and let $\bf{X}$ denote a matrix of potential predictors, including an intercept ...
2
votes
0answers
10 views

Best methods to integrate summed probability and uncertainty from multiple sources?

I am trying to calculate risk of coastal flooding. In our model, we have 4 major sources of uncertainty: Elevation at location (height + gaussian error) Sea level rise (value at any given year + ...
0
votes
0answers
8 views

How to be confident in the specificity of a threshold?

I am a student in a hospital department and looking back through ~5 years of testing I have found ~70 patients who received a test. I found a threshold which is the lowest result compatible with ...
0
votes
1answer
188 views

How to decide threshold values for AUC?

With respect to the pROC package for R(https://rdrr.io/cran/pROC/man/ggroc.html). ...
2
votes
0answers
42 views

Threshold crossing: Integrating posterior distribution over a forecast period

Suppose I have some time series $\{Y_t\}_{t=1}^T$ and I have come up with some probabilistic model: $$p(Y_{t+h} \mid Y_{1:t}, \theta)$$ for any horizon $h$, and some parameters $\theta$. Now ...
0
votes
0answers
98 views

How to define a threshold based on box-plot

I have a set of ordered number list as follows. 25789, 20500, 18888, 15000, 13200, ...., 118, 8, 1, 0 In my dataset most of the values are in the range of (1000-...
-1
votes
2answers
119 views

In elbow curve how to find the point from where the curve starts to rise? [closed]

I am computing a distance on my data. The result is then being sorted in ascending order. The samples having distance more than a specific threshold are to be marked as outliers and will be discarded. ...
1
vote
0answers
51 views

Optimal Threshold On Single Input

Suppose we have a sensor that measures "how red" an object is. Using this single input, we would like to classify whether or not the object is round. We have a data set comprised of tuples where the ...
3
votes
2answers
2k views

How to determine the optimal threshold to achieve the highest accuracy

I have a list of probabilities outputted by a classifier on a balanced dataset. The metric I want to maximize is accuracy ($\frac{TP+TN}{P+N}$). Is there a way to calculate the best threshold (without ...
0
votes
1answer
26 views

Measuring the effect of a variable across a threshold

Within my data I am trying to assess whether the response variable increases as we move across different thresholds. The difficulty is that the response variable also increases exponentially as a ...
0
votes
0answers
434 views

Precision/Recall against threshold curve not useful in improving model performance

I am doing fraud detection (binary classification) on an unbalanced credit dataset, using SVM. My current precision is 5% while recall is 90%. (I am using undersampling to train and test a model from ...
3
votes
0answers
76 views

MCMC in Gibbs: Threshold Multivariate Linear Regression [closed]

I have a problem regarding what I expected to be a rather straightforward modelling approach: I want to investigate multivariate linear regression parameters and a thresholding parameter which defines ...
0
votes
1answer
68 views

Does the threshold value of a logistic regression hypothesis has an effect on the accuracy?

It is true that the threshold value of a logistic regression hypothesis has an effect on the Precision/Recall metrics. Suppose you have trained a logistic regression classifier which is outputting $...
0
votes
1answer
819 views

How to implement thresholded softmax in Keras?

I would like to implement a threshold after the final softmax layer in a Keras-built classification problem so that class assignments with probability below some threshold alpha are disregarded (i.e. ...
14
votes
5answers
2k views

Philosophical question on logistic regression: why isn't the optimal threshold value trained?

Usually in logistic regression, we fit a model and get some predictions on the training set. We then cross-validate on those training predictions (something like here) and decide the optimal threshold ...
0
votes
0answers
43 views

What is the difference between finding the threshold and training a binary classifier?

I have 1-D data: the output of a Cosine similarity (COS) distance between two features. The data is bounded by [-1,1]. When I just try to find the optimum threshold for class labels - I get an ...
1
vote
1answer
708 views

Learn threshold for multi-label classification

I have a multi-label problem which I'm tackling with a NN. To get the multi-label scores, I use a tanh on the last layers (as suggested in the literature), and then selecting the ones corresponding to ...
2
votes
0answers
107 views

ROC curve for comparing probability of default models

I'm trying to compare two different probability of default models together by roc curve.I calculated the PD for 8 company by two different models.I know about the basic of roc curve and i can ...