Questions tagged [threshold]

Used (1) for discrete classification (if an instance's predicted probability exceeds a threshold, classify as TRUE, otherwise FALSE), or (2) for discretizing/binning continuous data. *If you are tempted to use this tag, PLEASE read the tag wiki!*

Filter by
Sorted by
Tagged with
0
votes
0answers
11 views

Materiality Threshold - statistics newbie (help!)

I am not sure how to approach the following situation and am looking for advice on how to develop a quantitative measure - The situation: I am comparing a rate calculated from two different sources, ...
2
votes
1answer
54 views

My machine learning models have reproducible results generally, but not at specific classification thresholds. What does this mean?

My binary classifier machine learning pipelines are producing generalizable results in that they are predicting the positive class (probability > .5) with the same precision on unseen data in ...
0
votes
0answers
29 views

What inference can we get from the frequency distribution of classification thresholds?

I've the probability scores of positive class of two models. The frequency distribution of those probability scores(thresholds) are like this Model #1 Model #2 It's a binary classification problem....
2
votes
2answers
47 views

Probability of Normal (Gaussian) random walk crossing threshold within k steps

Let $x[n]$ be a Gaussian random walk, so $x[0] = 0$ and $x[n+1] = x[n] + v$, where $v$ is an independent random variable with normal distribution, $0$ mean and standard deviation $s$. What is the ...
1
vote
1answer
25 views

ROC curves: can a cut-point (cut-off) be “useful”, or is it a term reserved to parameters only?

This is a semantic question: When analyzing Receiver Operating Characteric (ROC) curves, we talk about the "usefulness" of a parameter based on its Area Under the Curve (AUC). "Useful" here refers to ...
0
votes
0answers
10 views

Choosing Prior Distribution for AR coefficients in TAR/SETAR model

I am trying to choose the prior distribution for the AR coefficients for a general TAR model. I am tone between a truncated normal and uniform priors. Are there books or resources you can suggest for ...
7
votes
1answer
124 views

Decision rule as a hyper-parameter in LASSO

I have a question that is related to the following: Is decision threshold a hyperparameter in logistic regression? but would like some clarification. The general consensus is that the decision rule ...
0
votes
0answers
9 views

Nested Cross Validation: Choosing a Classification probability threshold

With nested CV: Inner loop for model selection, outer loop for performance evaluation. At what level can we optimize a threshold probability (vs. 0.5) to maximize sensitivity or specificity of the ...
0
votes
0answers
43 views

Tuning threshold from multiclass ROC for Gradient Boosting Classifier?

I have created a ROC curve based on the output of a multiclass Gradient Boosting Classifier (See Figure below implemented from Yellowbrick ROCAUC: https://www.scikit-yb.org/en/latest/api/classifier/...
0
votes
0answers
15 views

How do I perform data cleaning for threshold detection using method of limits?

I have a dataset collected using the Method of Limits (https://en.wikipedia.org/wiki/Psychophysics#Method_of_limits) and am looking for any approaches to outlier detection that are accepted practice. ...
1
vote
1answer
16 views

Determining break point/threshold in species richness in relation to land cover from multi-species occupancy models in R

I am conducting research in which I would like to see if I can detect a threshold response in mammal species richness in relation to the surrounding level of forest across study sites. I plan to use ...
-1
votes
1answer
27 views

R-Caret : Not-meaningful class probabilities and AUC value

I am very new to ML therefore my question might be primitive. I am working on a binary-class problem. The response (target) variable is occurrence : a factor ...
1
vote
1answer
36 views

Bayesian liability threshold model

Let $\bf{y}$ denote a vector of binary data, such as whether a group of individuals suffer from a particular disease, and let $\bf{X}$ denote a matrix of potential predictors, including an intercept ...
2
votes
0answers
8 views

Best methods to integrate summed probability and uncertainty from multiple sources?

I am trying to calculate risk of coastal flooding. In our model, we have 4 major sources of uncertainty: Elevation at location (height + gaussian error) Sea level rise (value at any given year + ...
0
votes
0answers
8 views

How to be confident in the specificity of a threshold?

I am a student in a hospital department and looking back through ~5 years of testing I have found ~70 patients who received a test. I found a threshold which is the lowest result compatible with ...
0
votes
1answer
56 views

How to decide threshold values for AUC?

With respect to the pROC package for R(https://rdrr.io/cran/pROC/man/ggroc.html). ...
1
vote
0answers
28 views

Threshold crossing: Integrating posterior distribution over a forecast period

Suppose I have some time series $\{Y_t\}_{t=1}^T$ and I have come up with some probabilistic model: $$p(Y_{t+h} \mid Y_{1:t}, \theta)$$ for any horizon $h$, and some parameters $\theta$. Now ...
0
votes
0answers
27 views

How to define a threshold based on box-plot

I have a set of ordered number list as follows. 25789, 20500, 18888, 15000, 13200, ...., 118, 8, 1, 0 In my dataset most of the values are in the range of (1000-...
-1
votes
2answers
47 views

In elbow curve how to find the point from where the curve starts to rise? [closed]

I am computing a distance on my data. The result is then being sorted in ascending order. The samples having distance more than a specific threshold are to be marked as outliers and will be discarded. ...
1
vote
0answers
40 views

Optimal Threshold On Single Input

Suppose we have a sensor that measures "how red" an object is. Using this single input, we would like to classify whether or not the object is round. We have a data set comprised of tuples where the ...
0
votes
0answers
11 views

Speed of transition parameter constraints

Given a logistic smooth transition regression \begin{equation} y_{t}=x_{t}^{\prime }\beta _{1}(1-{g}(z_{t};\gamma ,\delta ))+x_{t}^{\prime }\beta _{2}{g}(z_{t};\gamma ,\delta )+\varepsilon _{t}% \text{...
3
votes
2answers
795 views

How to determine the optimal threshold to achieve the highest accuracy

I have a list of probabilities outputted by a classifier on a balanced dataset. The metric I want to maximize is accuracy ($\frac{TP+TN}{P+N}$). Is there a way to calculate the best threshold (without ...
0
votes
1answer
23 views

Measuring the effect of a variable across a threshold

Within my data I am trying to assess whether the response variable increases as we move across different thresholds. The difficulty is that the response variable also increases exponentially as a ...
0
votes
0answers
15 views

Determining final TPR and FPR in staged classification model

I'm given pictorial dataset with 3 classes where I'm using the following scheme for classification (this scheme was chosen because classes 2 and 3 are close to each other, so this approach yields ...
0
votes
0answers
247 views

Precision/Recall against threshold curve not useful in improving model performance

I am doing fraud detection (binary classification) on an unbalanced credit dataset, using SVM. My current precision is 5% while recall is 90%. (I am using undersampling to train and test a model from ...
3
votes
0answers
51 views

MCMC in Gibbs: Threshold Multivariate Linear Regression [closed]

I have a problem regarding what I expected to be a rather straightforward modelling approach: I want to investigate multivariate linear regression parameters and a thresholding parameter which defines ...
0
votes
0answers
21 views

How can I use receiver operator curve in this SVM classification problem?

Short description of the learning task: I have a corpus containing voice segments annotated with the mean BPM obtained from heart rate recordings. For example, one sample would be like 5 s audio, ...
0
votes
0answers
18 views

how to parameter the gamess function?

i want to run gampackage to calculate threshold and semi parametric model, especialy partialy linear model like : Y = V1 + V2 + V3 + f(V4) where: Y is factor and dichotomic varible; V1 is continuous ...
0
votes
1answer
33 views

Does the threshold value of a logistic regression hypothesis has an effect on the accuracy?

It is true that the threshold value of a logistic regression hypothesis has an effect on the Precision/Recall metrics. Suppose you have trained a logistic regression classifier which is outputting $...
0
votes
1answer
498 views

How to implement thresholded softmax in Keras?

I would like to implement a threshold after the final softmax layer in a Keras-built classification problem so that class assignments with probability below some threshold alpha are disregarded (i.e. ...
0
votes
0answers
20 views

Placebo Thresholds in a Fuzzy Regression Discontinuity Design

Matsudaira (2008) used test scores to evaluate the impact of summer school attendance on students' performance. In this system, a student that scored less than an arbitrary grade was more likely to be ...
0
votes
0answers
24 views

How to find a threshold based on overlapping the histograms of two classes?

I want to perform thresholding to post-process the test data. I averaged the pixel intensity histograms of normal and abnormal cell images (grayscale, 8-bit, 256 * 256 images), overlapped them to ...
13
votes
5answers
2k views

Philosophical question on logistic regression: why isn't the optimal threshold value trained?

Usually in logistic regression, we fit a model and get some predictions on the training set. We then cross-validate on those training predictions (something like here) and decide the optimal threshold ...
0
votes
0answers
48 views

Threshold variable: Stationary?

Say I have an Logistic Smooth transition model (or even a "TAR" model): \begin{equation} \tilde{y}_{t}=x_{t}^{\prime }\beta _{1}(1-{g}(z_{t};\gamma ,\delta ))+x_{t}^{\prime }\beta _{2}{g}(z_{t};\gamma ...
0
votes
0answers
20 views

Justification for LST model

Let’s say I have a linear model specification but I believe that a logistic smooth transition model would be better. Is there any evidence I can take from the linear model that would point me in ...
0
votes
0answers
35 views

Multiclass classifier with undefined prediction, how to calculate metrices

I build a multiclass classifier. I want the classifier to predict a few samples with little false positives, rather then many samples with lots of false positives. Therefore I want to choose a ...
0
votes
0answers
14 views

Help inference results of a threshold regression model with multiple threshold variable

Just ran the package pdR, but i am having problems interpreting the results. Do i only check the p-values to validate results? what else do i have to look at? Output[[1]] #Formatted output of 1st ...
0
votes
0answers
30 views

Can I use logistic regression to calculate the optimal classification threshold between two class based on a single contiguous variable

I have a single contiguous variable, x, and there is two possible classes/labels y for n observations. I need to build a simple binary classifier with a threshold on x so that the number of ...
0
votes
0answers
27 views

What is the difference between finding the threshold and training a binary classifier?

I have 1-D data: the output of a Cosine similarity (COS) distance between two features. The data is bounded by [-1,1]. When I just try to find the optimum threshold for class labels - I get an ...
0
votes
1answer
369 views

Learn threshold for multi-label classification

I have a multi-label problem which I'm tackling with a NN. To get the multi-label scores, I use a tanh on the last layers (as suggested in the literature), and then selecting the ones corresponding to ...
2
votes
0answers
88 views

ROC curve for comparing probability of default models

I'm trying to compare two different probability of default models together by roc curve.I calculated the PD for 8 company by two different models.I know about the basic of roc curve and i can ...
2
votes
1answer
1k views

Adjust thresholds in multi-class classification [duplicate]

I have trained a random forest classifier on a (highly-imbalanced) 3-class problem (A 1% of the data, B 96%, C 3%) and obtained probabilities for each of the three classes. Currently I assign an ...
1
vote
0answers
57 views

Techniques for data thresholding

I had a query regarding one of the problems I am trying to solve. I have a few things in mind and would love if you could suggest and/ or point me in the right direction. Problem Statement - I have a ...
2
votes
1answer
72 views

Soft thresholding (Donoho and Johnstone)

Donoho and Johnstone (1994) poses the following equality: $$ E((\eta_t(X) - \mu)^2) = 1 - 2\Pr(|X|\lt t) + E(\min(X^2,t^2)) $$ where $\eta_t(X) = \operatorname{sign}(x)\max(|X|-t,0)$ and $X \sim N(\...
1
vote
1answer
244 views

Calculate threshold value for Poisson distributed noise

I need to calculate a threshold value to get rid of Poisson distributed noise in an image to perform a cluster analysis on the image. The image is the representation of a signal, whose datapoints ...
0
votes
0answers
357 views

Determine the cutoff threshold for binary classification models using cross validation

I have a binary classification model which returns class probability scores as the outcome. I did the model parameter tuning using repeated k-fold cross-validation. I am wondering how can I decide the ...
1
vote
0answers
41 views

Multiple outputs quantile regression

I am building a model to determine upper and lower thresholds for multiple features with respect to multiple others. (Specifically, I want to see if certain pressure measurements in an HVAC system ...
0
votes
1answer
2k views

Threshold to build confusion matrix?

I a have data set with 10 sections of data and each section shows one day observation. I designed the training and test set as follows: 8 sections for training the data and the last two sections for ...
1
vote
0answers
36 views

How to determine the optimal threshold for my classification problem using fpr ,tpr values for each classification record?

The output of my prediction using classification algorithm is in dataframe that contains TPR and FPR value for each prediction. Suppose I have 100 records for prediction then in that cade my data ...
1
vote
0answers
292 views

Implementing a threshold for multiclass classification with softmax activation

We have a CNN where images are classified into between 7 and 30 classes, depending on the training set. The final output is via SoftMax activation, and thus all the probabilities add to one. I notice ...