2k views

### Is accuracy a reliable evaluation metric for training a classification model? [duplicate]

What is the problem in training on accuracy when balancing classes during training? I Understand why accuracy is not a good metric for the test data evaluation of the model, as in the test data we ...
139 views

### Why to choose AUC over accuracy? [duplicate]

I am working on a fraud detection algorithm using a banking dataset which has large number of transactions. The number of true fraud cases are very small (<1%). So accuracy is not a good measure as ...
561 views

### High precision or High recall [duplicate]

I have a question about precision and recall. Which classifier is better, one with high precision or high recall for medical purposes like finding patients with allergy? Can we use F-measure for ...
88 views

### Accuracy, Sensitivity, Specificity, & ROC AUC [duplicate]

In the context of predictive modeling, when comparing clasification models, What statistic should be considered more important over the others: Accuracy, sensitivity, specificity, or area under ROC ...
140 views

### What is the meaning of AUC being high when accuracy is not? [duplicate]

I'm testing several classifiers in Weka Experimenter. Some of them have — at the same time — low accuracy (Percent_correct statistic) and high AUC. How should the quality of such ...
106 views

### Metrics for unbalanced classes [duplicate]

I have been looking for good metrics on this data set I am working, however it is highly unbalanced. It has a total of 8 categorical classes, one of them is responsible for ~40% of data, another for ...
65 views

### How good is a model if it can't predict a single positive class? [duplicate]

I have a training set of over a 100,000 points that is used to train a Logistic Regression Classifier (logit, since response is binary). The model is testing/fitted on a test set of 20,000 items. The ...
48 views

### Accuracy in multi-label classification [duplicate]

In a multi-label classification, the accuracy is commonly defined as [1]  \text{Accuracy}(\boldsymbol{Y},\, \boldsymbol{Z}) := \frac{1}{n} \sum_{i=1}^n \frac{\lvert Y_i \cap Z_i \rvert}{\lvert Y_i \...
40 views

### Reliability Logistic Regression - train and evaluate the model [duplicate]

I have built an Logistic Regression model in R. The class that I want to predict, is very unbalanced (99 vs 1). My first finding is that this Logistic model does a better job if I train it on a ...
38 views

### Training set target categories' distribution [duplicate]

In a book I'm reading I've come across the following quote: Accuracy on the test set is a good performance measure only when there is a relatively uniform distribution of target categories in the ...
28 views

### Average Precision or FBeta & Decision Threshold Tuning for Binary Classifier [duplicate]

I'm working with an imbalanced binary classifier data set (3% positive) in sklearn. The cost of a false negative is extremely high so recall is much more important than precision. To baseline my ...
22 views

### When is accuracy a good metric (as opposed to precision, recall, F1)? [duplicate]

Suppose you have a perfectly balanced data-set. In which applications is accuracy a good metric? Are there applications where it's preferable to precision, recall, and F1 (all at the same time)?
20 views

### Evaluating binary classifier model. What can say precision, recall etc.? [duplicate]

i'm trying to understand wether my model has good performance or not. I have binary classifier for summarization sentences: important or not (extractive approach) on specific corpus. Dataset is ...