Questions tagged [threshold]
Used (1) for discrete classification (if an instance's predicted probability exceeds a threshold, classify as TRUE, otherwise FALSE), or (2) for discretizing/binning continuous data. *If you are tempted to use this tag, PLEASE read the tag wiki!*
257
questions
4
votes
1
answer
386
views
My machine learning models have reproducible results generally, but not at specific classification thresholds. What does this mean?
My binary classifier machine learning pipelines are producing generalizable results in that they are predicting the positive class (probability > .5) with the same precision on unseen data in ...
- 135
3
votes
2
answers
454
views
Probability of Normal (Gaussian) random walk crossing threshold within k steps
Let $x[n]$ be a Gaussian random walk, so $x[0] = 0$ and $x[n+1] = x[n] + v$, where $v$ is an independent random variable with normal distribution, $0$ mean and standard deviation $s$.
What is the ...
- 31
1
vote
1
answer
68
views
ROC curves: can a cut-point (cut-off) be "useful", or is it a term reserved to parameters only?
This is a semantic question:
When analyzing Receiver Operating Characteric (ROC) curves, we talk about the "usefulness" of a parameter based on its Area Under the Curve (AUC). "Useful" here refers to ...
- 125
10
votes
1
answer
901
views
Decision rule as a hyper-parameter in LASSO
I have a question that is related to the following: Is decision threshold a hyperparameter in logistic regression?
but would like some clarification.
The general consensus is that the decision rule ...
- 1,488
1
vote
1
answer
122
views
Determining break point/threshold in species richness in relation to land cover from multi-species occupancy models in R
I am conducting research in which I would like to see if I can detect a threshold response in mammal species richness in relation to the surrounding level of forest across study sites. I plan to use ...
- 11
2
votes
1
answer
189
views
Bayesian liability threshold model
Let $\bf{y}$ denote a vector of binary data, such as whether a group of individuals suffer from a particular disease, and let $\bf{X}$ denote a matrix of potential predictors, including an intercept ...
- 813
2
votes
0
answers
20
views
Best methods to integrate summed probability and uncertainty from multiple sources?
I am trying to calculate risk of coastal flooding. In our model, we have 4 major sources of uncertainty:
Elevation at location (height + gaussian error)
Sea level rise (value at any given year + ...
- 5,183
0
votes
1
answer
1k
views
How to decide threshold values for AUC?
With respect to the pROC package for R(https://rdrr.io/cran/pROC/man/ggroc.html).
...
2
votes
0
answers
76
views
Threshold crossing: Integrating posterior distribution over a forecast period
Suppose I have some time series $\{Y_t\}_{t=1}^T$ and I have come up with some probabilistic model:
$$p(Y_{t+h} \mid Y_{1:t}, \theta)$$
for any horizon $h$, and some parameters $\theta$.
Now ...
- 1,320
-1
votes
2
answers
519
views
In elbow curve how to find the point from where the curve starts to rise? [closed]
I am computing a distance on my data. The result is then being sorted in ascending order.
The samples having distance more than a specific threshold are to be marked as outliers and will be discarded. ...
- 153
1
vote
0
answers
80
views
Optimal Threshold On Single Input
Suppose we have a sensor that measures "how red" an object is. Using this single input, we would like to classify whether or not the object is round. We have a data set comprised of tuples where the ...
- 241
5
votes
3
answers
6k
views
How to determine the optimal threshold to achieve the highest accuracy
I have a list of probabilities outputted by a classifier on a balanced dataset. The metric I want to maximize is accuracy ($\frac{TP+TN}{P+N}$). Is there a way to calculate the best threshold (without ...
- 153
0
votes
1
answer
84
views
Measuring the effect of a variable across a threshold
Within my data I am trying to assess whether the response variable increases as we move across different thresholds. The difficulty is that the response variable also increases exponentially as a ...
2
votes
0
answers
2k
views
Precision/Recall against threshold curve not useful in improving model performance
I am doing fraud detection (binary classification) on an unbalanced credit dataset, using SVM. My current precision is 5% while recall is 90%. (I am using undersampling to train and test a model from ...
- 21
3
votes
0
answers
114
views
MCMC in Gibbs: Threshold Multivariate Linear Regression [closed]
I have a problem regarding what I expected to be a rather straightforward modelling approach: I want to investigate multivariate linear regression parameters and a thresholding parameter which defines ...
- 1,322
0
votes
1
answer
924
views
Does the threshold value of a logistic regression hypothesis has an effect on the accuracy?
It is true that the threshold value of a logistic regression hypothesis has an effect on the Precision/Recall metrics.
Suppose you have trained a logistic regression classifier which is outputting $...
- 103
0
votes
1
answer
2k
views
How to implement thresholded softmax in Keras?
I would like to implement a threshold after the final softmax layer in a Keras-built classification problem so that class assignments with probability below some threshold alpha are disregarded (i.e. ...
17
votes
5
answers
3k
views
Philosophical question on logistic regression: why isn't the optimal threshold value trained?
Usually in logistic regression, we fit a model and get some predictions on the training set. We then cross-validate on those training predictions (something like here) and decide the optimal threshold ...
- 676
0
votes
0
answers
74
views
What is the difference between finding the threshold and training a binary classifier?
I have 1-D data: the output of a Cosine similarity (COS) distance between two features. The data is bounded by [-1,1].
When I just try to find the optimum threshold for class labels - I get an ...
- 1
1
vote
1
answer
2k
views
Learn threshold for multi-label classification
I have a multi-label problem which I'm tackling with a NN. To get the multi-label scores, I use a tanh on the last layers (as suggested in the literature), and then selecting the ones corresponding to ...
- 161
2
votes
1
answer
242
views
ROC curve for comparing probability of default models
I'm trying to compare two different probability of default models together by roc curve.I calculated the PD for 8 company by two different models.I know about the basic of roc curve and i can ...
- 41
3
votes
1
answer
8k
views
Adjust thresholds in multi-class classification [duplicate]
I have trained a random forest classifier on a (highly-imbalanced) 3-class problem (A 1% of the data, B 96%, C 3%) and obtained probabilities for each of the three classes.
Currently I assign an ...
- 311
1
vote
0
answers
64
views
Techniques for data thresholding
I had a query regarding one of the problems I am trying to solve. I have a few things in mind and would love if you could suggest and/ or point me in the right direction.
Problem Statement - I have a ...
- 121
2
votes
1
answer
182
views
Soft thresholding (Donoho and Johnstone)
Donoho and Johnstone (1994) poses the following equality:
$$
E((\eta_t(X) - \mu)^2) = 1 - 2\Pr(|X|\lt t) + E(\min(X^2,t^2))
$$
where $\eta_t(X) = \operatorname{sign}(x)\max(|X|-t,0)$ and $X \sim N(\...
- 133
1
vote
1
answer
479
views
Calculate threshold value for Poisson distributed noise
I need to calculate a threshold value to get rid of Poisson distributed noise in an image to perform a cluster analysis on the image.
The image is the representation of a signal, whose datapoints ...
- 13
0
votes
0
answers
599
views
Determine the cutoff threshold for binary classification models using cross validation
I have a binary classification model which returns class probability scores as the outcome. I did the model parameter tuning using repeated k-fold cross-validation. I am wondering how can I decide the ...
- 113
1
vote
0
answers
238
views
Multiple outputs quantile regression
I am building a model to determine upper and lower thresholds for multiple features with respect to multiple others. (Specifically, I want to see if certain pressure measurements in an HVAC system ...
- 186
0
votes
1
answer
6k
views
Threshold to build confusion matrix?
I a have data set with 10 sections of data and each section shows one day observation. I
designed the training and test set as follows: 8 sections for training the data and the last two sections for ...
- 109
1
vote
0
answers
40
views
How to determine the optimal threshold for my classification problem using fpr ,tpr values for each classification record?
The output of my prediction using classification algorithm is in dataframe that contains TPR and FPR value for each prediction. Suppose I have 100 records for prediction then in that cade my data ...
- 111
1
vote
0
answers
681
views
Implementing a threshold for multiclass classification with softmax activation
We have a CNN where images are classified into between 7 and 30 classes, depending on the training set. The final output is via SoftMax activation, and thus all the probabilities add to one.
I notice ...
- 211
0
votes
1
answer
297
views
Statistical test for count data passing a given threshold
Apologies if this question is painfully basic. I have a df of count data in which each row represents a box of marbles, some of which are black, and some are white:
...
- 117
0
votes
0
answers
159
views
How to create a binary threshold from Cosine distance between 1-D arrays?
I have a graph of the Cosine distance between the question and the sentence most similar to it when there is an answer and when there is none.
I want to establish a threshold on the abscissa axis ...
1
vote
0
answers
77
views
Whether and how to bin data beyond a certain threshold
I've seen many admonishments against binning and agreed with them in general, but I wonder it's advisable to do so under certain circumstances.
To adapt an example from my actual project, let's say ...
- 111
1
vote
1
answer
472
views
General rule uniform distributed classes
Given a classifier working with double values e.g. between 0 and 1. There are two classes with different ranges. Their distributions are uniform, however, one class is more likely.
Is picking always ...
- 631
0
votes
3
answers
430
views
Statistically prove classification accuracy is acceptable
I made a neural network to do binary classification with medical images. Since I evaluate test accuracy (x% accurate), is there any statistical method which can be ...
1
vote
0
answers
674
views
Practically speaking, is the TF-IDF threshold universal across different corpus?
I would like to know the practical threshold of the TF-IDF (just like the practical p-value cutoff of 0.1 or 0.05 in hypothesis tests).
I tried to look at it in some previous post, and some people ...
- 133
2
votes
0
answers
37
views
estimating Threshold-Models in seperate equations
I´m currently dealing with threshold VECM models in time series. The basic idea behind a TVECM, according to Lo and Zivot 2001 is
There is one point I don´t understand: As far as I see every ...
- 542
0
votes
1
answer
40
views
Establish the proper prediction threshold
Suppose for a ML classification problem X, I have three labels, i.e. 1, 0 and -1. For some reasons, I obtained bad accuracy. So I decided to change the number of labels to two, i.e. 1 and -1. That way ...
- 101
0
votes
1
answer
1k
views
Finding an optimal class probability threshold for SVM
I've got an imbalanced data set on which I'm training an SVM using cross validation. I'd like to find the optimal class probability threshold that maximizes the F measure.
I've tried doing this by ...
- 3
4
votes
0
answers
108
views
What is the soft and the hard thresholding operator?
I'm trying to understand in simple words what the soft and hard thresholding operators are and what they do?
I looked some examples on the internet but I can't find anything simple.
Any suggestion ...
- 827
2
votes
1
answer
2k
views
How many missing values in a dataset can Xgboost handle?
I have a dataset with a lot of missing values in the columns. I dropped all columns containing more than 70% of missing values.
I'm considering using Xgboost for my prediction because it can handle ...
- 25
1
vote
0
answers
69
views
Decision rule for Bayesian variable selection
I'm fitting a spike-and-slab regression model to sparse data and obtaining posterior probabilities of association (PPA). I now would like to 'declare' associations based on the latter. I'm concerned ...
- 405
1
vote
0
answers
92
views
Find a cost-sensitive multiclass algorithm
I am working on decision trees which can directly work in a multiclass context. My aim is reduce the misclassification's errors of a decision tree by improving its ability to tackle imbalanced ...
- 11
-1
votes
1
answer
520
views
Constructing feature vectors for clustering from row values
Thank you for some ideas regarding my question below.
Background:
I have an excel file with each row representing transactions (or transaction id's) in the format below and I am creating a feature ...
- 179
0
votes
0
answers
199
views
Difference between using thresholds to classify and using a custom objective function
Suppose we are faced with a binary classification problem. The standard approach seems to be to fit a probability estimator using a loss function like log loss and then afterwards determine the ...
- 43
2
votes
0
answers
697
views
Ordered Probit Model in R: Latent Variable and Threshold Parameters
I have recently started to improve my methodological skills and programming in R. For a study term paper I now want to run an ordered logit/probit model.
The data I use are taken from the European ...
- 21
2
votes
0
answers
393
views
What is the correct way to modify the bin-counts given a threshold for a chi-squared test?
When performing a chi-square test, one inputs the expected counts (via integrating probability distribution over respective bin bounds) and observed counts into the chi-square formula (denoted below).
...
user146123
4
votes
0
answers
785
views
How to choose threshold in probabilistic multi-label classification?
My problem is to tag some texts with some labels. That is multi-label classification in ML.
I did predict with Catboost classifier and get some results of binary ...
- 447
5
votes
1
answer
4k
views
Training threshold vs validation threshold for better prediction results?
Between the two, should I use a model's training or validation threshold to get best results (from a distributed random forest binary classifier built using h2o.ai) ...
2
votes
2
answers
949
views
Probability threshold and signal/noise ratio
I'm working on a classification problem in which the underlying signal to identify is very hard to find.
I suppose that this is because the signal/noise ratio is very low.
My questions are ...
- 762