# Questions tagged [f1]

a popular criterion for evaluating binary decision algorithms and classification models.

71 questions
Filter by
Sorted by
Tagged with
1 vote
14 views

### Is there an equivalent for Yates' correction for a confusion matrix-derived metrics?

Given the following table of predictions vs. actual states: ...
11 views

### Proving stability of F1 metric for a given sample size

Okay so say you have sequence classification problem: extracting entities from conversations. Say one of the labels is CITIES. Say you have calculated P/R/f1/support for CITIES and it looks like this: ...
27 views

### Binary classification metrics - Combining sensitivity and specificity?

The harmonic mean between precision and recall (F1 score) is a common metric to evaluate binary classification. It is useful because it strikes a balance between precision (FP) and recall (FN). For ...
29 views

### Correctness of derivation for binary F1 variance for F1 confidence intervals

I'm developing a python library for confidence intervals for common accuracy metrics, with both analytic and bootstrap computations. Following this paper, I implemented the Macro and Micro F1 scores ...
49 views

### Inverse Weighted Average F1-Score

I am dealing with a binary classification problem (class 0/1) with class imbalance. Given the vector of predictions, I would like to compute: F1-Score for class 0 F1-Score for class 1 Weighted ...
1 vote
68 views

### Why don't we use the harmonic mean of sensitivity and specificity?

There is this question on the F-1 score, asking why we compute the harmonic mean of precision and recall rather than its arithmetic mean. There were good arguments in the answers in favor of the ...
286 views

### F1 score for validation and testing datasets is different

I have the following F1 score function that I use for the model when I train it as part of metrics and as well during prediction: ...
274 views

### Calculating the Brier or log score from the confusion matrix, or from accuracy, sensitivity, specificity, F1 score etc

Suppose I have a confusion matrix, or alternatively any one or more of accuracy, sensitivity, specificity, recall, F1 score or friends for a binary classification problem. How can I calculate the ...
1k views

### Academic reference on the drawbacks of accuracy, F1 score, sensitivity and/or specificity

Accuracy, as a KPI for assessing binary classification models, has major drawbacks: Why is accuracy not the best measure for assessing classification models?. The exact same issues also plague the F1 ...
1 vote
41 views

### Statistical significance of performance difference in classification models

Is it possible to assign a p-value to the mean performance difference in three classification models? The models use the same data, same random seed, and use 10-fold cross validation. Model A has a ...
106 views

### Relationship between precision-recall curve and Fmax

These two metrics are both usually appropriate for imbalanced classification. Since $$F_{max} = \max{\frac{2\cdot precision\cdot recall}{precision+recall}},$$ I'm guessing Fmax might be somewhat "...
1 vote
137 views

### Confidence Interval of the Average of a F1 Score Samples

I have a number of individual F1 score samples and right now I am measuring the average F1 score across this group. However, I would also like to present a confidence interval on it. Its a continuous ...
126 views

### Singular beta in the F-beta vs. threshold score?

Consider this plot of the $F_\beta$ score for different values of $\beta$. I have a hard time getting an intuition as to why they intersect at a same point. (Cf. this blog post.) In other words, why ...
1 vote
39 views

### Optimal metric for training with Class-specific masked input features and imbalanced dataset

I have a classification problem of 8-classes, which are extremely imbalanced. The input dataset consists of sequences, each of length n features, where n = 19. For each of the 8 classes, I have a ...
43 views

### Cross-validation and F1 metric

Cross-validation with metrics such as F1 can be implemented in two ways: For each cross-validation split, calculate F1_split on the validation dataset. F1_result = average_by_splits(F1_split) For ...
46 views

### Harmonic mean of false positive and false discovery rates (analogous to F1)

F1 is the harmonic mean of recall (aka sensitivity, or true positive rate, TPR) and precision (aka positive predictive value, PPV). $\text{TPR} = \text{Pr(predicted:Pos | Pos)} =$ TP/P (wikipedia ...
753 views

### Selecting best classification probability threshold with ROC/AUC doesn't necessarily improve F1 score

I read that probability based binary classifiers have 0.5 as default probability threshold for getting hard 0/1 labels (in scikit-learn for example) but this could be fine-tuned with methods like ...
1 vote
97 views

950 views

### Are Jaccard score and F1 score monotonically related?

I have compared the rankings obtained by comparing 10+ classifiers with this two metrics: Jaccard score F1 score They show a perfect correlation. This results holds on 50+ datasets. When comparing ...
1 vote
2k views

### Are F1 score and Dice coefficient computed in same way or different way in image segmentation (two class segmentation)?

On page 8 of the paper An automatic nuclei segmentationmethod based on deep convolutional neuralnetworks for histopathology images, the authors show performance of their deep model on test sets. They ...
1 vote
803 views

### F1 weighted vs. Log loss in SciKit learn RandomSearchCV

I am sorry to ask another question regarding this topic but I am still puzzled about the following: When I use 'F1_weighted' as my scoring argument in a ...