Linked Questions

0
votes
2answers
2k views

Is accuracy a reliable evaluation metric for training a classification model? [duplicate]

What is the problem in training on accuracy when balancing classes during training? I Understand why accuracy is not a good metric for the test data evaluation of the model, as in the test data we ...
1
vote
2answers
139 views

Why to choose AUC over accuracy? [duplicate]

I am working on a fraud detection algorithm using a banking dataset which has large number of transactions. The number of true fraud cases are very small (<1%). So accuracy is not a good measure as ...
1
vote
1answer
561 views

High precision or High recall [duplicate]

I have a question about precision and recall. Which classifier is better, one with high precision or high recall for medical purposes like finding patients with allergy? Can we use F-measure for ...
1
vote
3answers
88 views

Accuracy, Sensitivity, Specificity, & ROC AUC [duplicate]

In the context of predictive modeling, when comparing clasification models, What statistic should be considered more important over the others: Accuracy, sensitivity, specificity, or area under ROC ...
0
votes
0answers
140 views

What is the meaning of AUC being high when accuracy is not? [duplicate]

I'm testing several classifiers in Weka Experimenter. Some of them have — at the same time — low accuracy (Percent_correct statistic) and high AUC. How should the quality of such ...
1
vote
2answers
106 views

Metrics for unbalanced classes [duplicate]

I have been looking for good metrics on this data set I am working, however it is highly unbalanced. It has a total of 8 categorical classes, one of them is responsible for ~40% of data, another for ...
0
votes
1answer
65 views

How good is a model if it can't predict a single positive class? [duplicate]

I have a training set of over a 100,000 points that is used to train a Logistic Regression Classifier (logit, since response is binary). The model is testing/fitted on a test set of 20,000 items. The ...
1
vote
0answers
48 views

Accuracy in multi-label classification [duplicate]

In a multi-label classification, the accuracy is commonly defined as [1] $$ \text{Accuracy}(\boldsymbol{Y},\, \boldsymbol{Z}) := \frac{1}{n} \sum_{i=1}^n \frac{\lvert Y_i \cap Z_i \rvert}{\lvert Y_i \...
0
votes
0answers
40 views

Reliability Logistic Regression - train and evaluate the model [duplicate]

I have built an Logistic Regression model in R. The class that I want to predict, is very unbalanced (99 vs 1). My first finding is that this Logistic model does a better job if I train it on a ...
0
votes
0answers
38 views

Training set target categories' distribution [duplicate]

In a book I'm reading I've come across the following quote: Accuracy on the test set is a good performance measure only when there is a relatively uniform distribution of target categories in the ...
0
votes
0answers
28 views

Average Precision or FBeta & Decision Threshold Tuning for Binary Classifier [duplicate]

I'm working with an imbalanced binary classifier data set (3% positive) in sklearn. The cost of a false negative is extremely high so recall is much more important than precision. To baseline my ...
1
vote
0answers
22 views

When is accuracy a good metric (as opposed to precision, recall, F1)? [duplicate]

Suppose you have a perfectly balanced data-set. In which applications is accuracy a good metric? Are there applications where it's preferable to precision, recall, and F1 (all at the same time)?
0
votes
0answers
20 views

Evaluating binary classifier model. What can say precision, recall etc.? [duplicate]

i'm trying to understand wether my model has good performance or not. I have binary classifier for summarization sentences: important or not (extractive approach) on specific corpus. Dataset is ...
0
votes
0answers
14 views

When do we use precision/recall over accuracy when evaluating a classifier? [duplicate]

When do we use precision/recall over accuracy when evaluating a classifier? What kind of scenarios would mean precision/recall metrics would be better than accuracy?
0
votes
0answers
13 views

Which performance metric to use for stratified data? [duplicate]

I'm trying to classify a data into 3 classes (supervised), one of which is heavily underrepresented in the data set. In order to combat this imbalance, I decided to stratify the data. Now I want to ...

15 30 50 per page