Linked Questions

2
votes
1answer
990 views

xgboost prediction threshold [duplicate]

I am trying to classify the data set "Insurance Company Benchmark (COIL 2000) Data Set" which can be found in Dataset. I am using XGBoost in R (I am new to XGBoost algorithm) for the classification ...
1
vote
1answer
838 views

Adjust thresholds in multi-class classification [duplicate]

I have trained a random forest classifier on a (highly-imbalanced) 3-class problem (A 1% of the data, B 96%, C 3%) and obtained probabilities for each of the three classes. Currently I assign an ...
3
votes
1answer
561 views

Logistic regression and classification: Adjusting or removing decision boundaries [duplicate]

I'm taking Andrew Ng's Machine Learning Course. In the lesson on classification algorithms, he presents the logit function ($\frac{1}{1+e^{-x}}$) and the way it converts parameterized functions to ...
1
vote
1answer
79 views

AUC ROC Threshold Setting in heavy imbalance [duplicate]

I am doing binary logistic regression on a dataset with very heavy class imbalance. Class 1 is only 1% of data. When I train logistic regressor without class weights I get ROC AUC Score of 0.6269. ...
1
vote
0answers
23 views

How can we best utilize the knowledge of P(y=1) in classification? [duplicate]

Premise I saw an interesting example of a machine learning logistic classifier for modeling/predicting sentiment for customer reviews. One of the first things in the example was a note on ...
0
votes
0answers
17 views

In what step should I try to find a best thread cut-off point for binary classification? [duplicate]

I am working on an imbalanced binary classification and wondering in what step I should find the best optimal threshold cut-off point. When I tried classifying the dataset with the normal probability ...
112
votes
7answers
38k views

Why is accuracy not the best measure for assessing classification models?

This is a general question that was asked indirectly multiple times in here, but it lacks a single authoritative answer. It would be great to have a detailed answer to this for the reference. ...
37
votes
8answers
23k views

What is the difference between prediction and inference?

I'm reading through "An Introduction to Statistical Learning" . In chapter 2, they discuss the reason for estimating a function $f$. 2.1.1 Why Estimate $f$? There are two main reasons we may ...
13
votes
5answers
2k views

Philosophical question on logistic regression: why isn't the optimal threshold value trained?

Usually in logistic regression, we fit a model and get some predictions on the training set. We then cross-validate on those training predictions (something like here) and decide the optimal threshold ...
13
votes
1answer
13k views

What is F1 Optimal Threshold? How to calculate it?

I've used h2o.glm() function in R which gives a contingency table in the result along with other statistics. The contingency table is headed "Cross Tab based on F1 Optimal Threshold" Wikipedia ...
13
votes
2answers
2k views

Is decision threshold a hyperparameter in logistic regression?

Predicted classes from (binary) logistic regression are determined by using a threshold on the class membership probabilities generated by the model. As I understand it, typically 0.5 is used by ...
8
votes
3answers
1k views

How to make the randomforest trees vote decimals but not binary

My question is about binary classification, say separating good customers from bad customers, but not regression or non-binary classification. In this context, a random forest is an ensemble of ...
8
votes
2answers
839 views

Example when using accuracy as an outcome measure will lead to a wrong conclusion

I am looking into various performance measures for predictive models. A lot was written about problems of using accuracy, instead of something more continuous to evaluate model performance. Frank ...
12
votes
1answer
1k views

Is accuracy an improper scoring rule in a binary classification setting?

I have recently been learning about proper scoring rules for probabilistic classifiers. Several threads on this website have made a point of emphasizing that accuracy is an improper scoring rule and ...
7
votes
2answers
1k views

How does one most easily overfit?

This is a weird question, I know. I'm just a noob and trying to learn about different classifier options and how they work. So I'm asking the question: Given a dataset of n1-dimensions and n2-...

15 30 50 per page