Questions tagged [f1]
a popular criterion for evaluating binary decision algorithms and classification models.
58
questions
1
vote
1
answer
20
views
F1 score for change point detection
I'm trying to evaluate my change point detection algorithm based on F1 score[1] defined as follows
Let $\mathcal{X}$ denote the set of change point locations provided by a detection algorithm and let $...
- 279
0
votes
0
answers
16
views
How to compute F1-score when there are NaN in the vectors?
I am using the f1-score (f1 = 2 * (precision * recall) / (precision + recall) to compute the similarity between two vectors (let's call them actual and pred). There are nevertheless some missing ...
- 1
1
vote
0
answers
31
views
combine accuracy precision and recall
I am working on a classification problem. Several models are produced and all have accuracy, precision and recall metrics on test data. I need to pick the best model among the alternatives.
What I can ...
- 31
0
votes
0
answers
17
views
How good do accuracy, precision, recall, and F1 have to be to use for flight critical environments?
How good do accuracy, precision, recall, and F1 have to be to use for flight critical environments?
Maybe there are rules of thumb, general practices, best practices, lessons learned, etc.
Flight ...
- 101
0
votes
1
answer
16
views
What is the procedure to find the optimal decision threshold in an imbalanced classification problem to maximize F1 score?
What is the procedure to find the optimal decision threshold in an imbalanced classification problem to maximize the F1 score? I'm using an xgboost model.
Your help is highly appreciated.
- 203
2
votes
1
answer
58
views
Why is "balanced accuracy" an arithmetic mean instead of harmonic?
In the F1 score, the harmonic mean of precision (Positive Predictive Value) and sensitivity/recall (True Positive Rate), I understand that we use the harmonic mean in order to penalize extreme values ...
- 11.7k
2
votes
1
answer
50
views
When to use f1_macro vs f1_weighted? [closed]
My dataset is imbalanced. The labels are binary. I'm training 2 logistic regression models.
For the first one, I didnt balance the model with class_weight and used ...
0
votes
0
answers
72
views
What to do with 99% F1 score in binary classification?
I've been handed a binary classification model to look after. The model uses the F1 score for comparison purposes. The challenge is that the F1 score against the test dataset is very high, like 99%, ...
- 111
0
votes
0
answers
54
views
class_weight='balanced' vs high f_beta score for imbalanced logistic regression in sklearn. Please help explain the difference
I have an imbalanced binary classification problem I am trying to solve with the LogisticRegression algorithm in sklearn.
As the data is highly imbalanced I am looking at ways to treat the imbalance ...
0
votes
1
answer
46
views
Why True Positives equals to True Negatives while calculating micro F1 or accuracy for multi-class single label classification?
I was searching what is the difference between micro F1 and accuracy since sklearn classification_report shows accuracy in place of micro F1.
I found multiple ...
- 121
0
votes
2
answers
110
views
What would be the statistically relevant patterns among $25, 18, 15, 12, 10, 8, 6, 4, 2, 1, =101{\it ?}$
Here is the current Formula One World Championship points scoring systems—
Currently in Formula 1, there is a lot of talk about this and whether it is adequate or if it should be changed.
$$\begin{...
0
votes
0
answers
36
views
Does a single F-score make sense for multiclass classification?
I'm newish to ML and trying to compute some performance metrics for a K-means classifier. I've constructed the following confusion matrix, and computed the per-class precision, recall, and F-scores (...
- 143
0
votes
0
answers
30
views
Is there an explanation for a classifier achieving high F1 scores, but having still high CrossEntropyLoss?
I am training a CNN classifier on a balanced dataset (around 35k examples for each label) with 13 classes. The model seems to achieve high F1 scores from the first batches; The F1 score for each class ...
- 3
0
votes
0
answers
52
views
Extending F1 score to classification with uncertainty
I have a classifier that can return yes, no's and "maybe"s.
The maybe indicates that we don't have certainty in a prediction because the data point is is not sufficiently close to it's ...
- 131
0
votes
0
answers
123
views
Can you calculate recall/precision/f1 score for "continuous variables"
My goal is to calculate some performance metrics (precision, recall, f1,accuracy) on a binary variable (yes or no) but have dates associated with each data point. Attaching dates to this binary ...
- 1
1
vote
0
answers
264
views
How to describe f1 soft loss mathematically
Normal F1-score using binarized prediction can be described like this:
$$F_1 = \frac{2 \cdot TP }{2 \cdot TP + FP + FN}$$
But in a loss function for a Machine Learning model, you will typically need ...
- 189
0
votes
0
answers
12
views
Reporting multiclass macro R/P/F1 when classes are absent from real world data
I have developed a hierarchical classification framework for my current project using a 'local classifier per parent node approach' (as described in https://link.springer.com/article/10.1007/s10618-...
1
vote
1
answer
295
views
How to calculate F1, Precision, and Recall for Multi-Label Multi-Classification
I have a predictive model as follows
Sample1
Sample2
Sample3
Sample4
Red
Yellow
Blue
Green
White
Black
Orange
65
21
55
40
0
0
1
0
1
0
0
31
40
44
30
0
0
0
0
0
0
0
33
44
56
66
1
0
0
1
0
0
1
63
77
...
- 251
1
vote
1
answer
84
views
Model accuracy versus F1
When training a model (classifier) in TensorFlow, an accuracy value is returned. What is the interpretation of an accuracy of, say, ...
- 111
0
votes
0
answers
59
views
Is there a F1-score for the negative class?
I am analysing some data and calculated the $F1 = \frac{2TP}{2TP + FP + FN}$
I'd like to know if there is a name to a negative-class based F-score, something like $F0 = \frac{2TN}{2TN + FP + FN} = 2 \...
- 101
0
votes
0
answers
56
views
How to calculate chance-level f1, ROC-AUC, PR-AUC for imbalanced dataset
I have an imbalance dataset (60% class 1, 40% class 0).
I trained a model and got accuracy, f1, ROC-AUC and PR-AUC.
I want to compare them to chance-level performance.
obviously chance-level of acc if ...
- 27
0
votes
2
answers
198
views
What is the F1 Score for my prediction when all values are negative?
I have built a model that gives me classification of some cases
here is a comparison between Actual and Prediction
...
- 251
0
votes
0
answers
19
views
How to evaluate F1 performances using sklearn for imbalanced multiclass classifier, with different weighting for train and test sets?
I have an imbalanced dataset. The number of occurrences\weight of each class in the test and train sets is different. I wish to use the sklearn implementation of the F1 score for the evaluation. I am ...
- 73
1
vote
1
answer
157
views
F1 Score is giving good value in imbalanced dataset
If I have an imbalanced dataset that consists of 90% positive points and 10% negative points. Now I created a "dumb" model which always predicts every point as a positive point. The ...
- 55
0
votes
0
answers
19
views
Continuously occurring true negatives
How can I handle discrete events in a continuous time stream in the context of an F1 metric?
To give an example, let's say the Earthquake Forecasting Bureau would report the following for their ...
- 1
2
votes
0
answers
103
views
Average Precision vs average F1
Average precision computes the area under the recall-precision curve by the trapezoidal rule (or midpoint rule). However, we could also compute the F1 score for every threshold and then take the ...
- 508
0
votes
1
answer
30
views
hyperparameter search with unknown test set distribution
I'm training a 3-class neural network classifier (conv layers and softmax at the end, nothing special). Let's say, in the test set I will have N1 examples of the 1st class, N2 examples of the 2nd ...
- 41
4
votes
1
answer
189
views
Why use harmonic mean for precision and recall (f1 score) instead of just the product of precision and recall?
General question here, I understand the purpose of using the harmonic mean to generate the f1 score for model evaluation. I'm not exactly sure though why we don't just take the product of precision ...
- 149
2
votes
1
answer
2k
views
On which set (train/val/test) do people calculate F1 score, precision and recall?
This may be a stupid question, but when I was looking at the definition of precision/recall etc. it was not mentioned anywhere which set (training/validation/test) this metric should be calculated ...
- 143
0
votes
0
answers
29
views
How to explain a relationship between Accuracy and F1 Score / F-Measure?
I am building a CNN model for pitch estimation using a song recording. For the evaluation metrics, I am using Accuracy and ...
0
votes
0
answers
769
views
f1-score of imbalanced data within k fold cross validation
I am trying to find the f1 score, precision, recall of a highly imbalanced dataset. I would like to use k-fold cross validation approach. I followed the procedure:
create arrays to store testing data ...
- 115
2
votes
1
answer
83
views
Harmonic is used in F1 score because it is a conservative metric: How does it help being conservative?
I was reading Jurafsky 3rd edition, page 12-13 chapter 4
Can you explain why is it good to weigh more the smaller of the two items namely $\frac{1}{Precision}$ or $\frac{1}{Recall}$?
Here is the link ...
- 279
1
vote
1
answer
416
views
Are Jaccard score and F1 score monotonically related?
I have compared the rankings obtained by comparing 10+ classifiers with this two metrics:
Jaccard score
F1 score
They show a perfect correlation. This results holds on 50+ datasets. When comparing ...
- 121
1
vote
0
answers
1k
views
Are F1 score and Dice coefficient computed in same way or different way in image segmentation (two class segmentation)?
On page 8 of the paper An automatic nuclei segmentationmethod based on deep convolutional neuralnetworks for histopathology images, the authors show performance of their deep model on test sets. They ...
- 111
1
vote
1
answer
509
views
F1 weighted vs. Log loss in SciKit learn RandomSearchCV
I am sorry to ask another question regarding this topic but I am still puzzled about the following:
When I use 'F1_weighted' as my scoring argument in a ...
- 161
6
votes
1
answer
1k
views
F1 score, PR or ROC curve for regression
Due to my background as a pure biologist, I've been struggling with the comment acquired from a reviewer about the accuracy test used in my regression study. While I stick to MSE, MAE and R2 as the ...
- 83
0
votes
1
answer
37
views
Is it possible for a binary classifier to have lower accuracy, macrof1 and binaryf1 but higher ROC AUC? [duplicate]
I've got the results of two classifiers based on 5 different splits of training and testing sets. Their mean and std of the results are as follow:
Method-------Accuracy -- MacroF1 -- BinaryF1---- ROC ...
- 131
0
votes
0
answers
403
views
Imbalanced classification what is good F1 score?
For imbalanced classification (say 85:15), what is good value of F1 score? An answer https://stats.stackexchange.com/a/217343/285091 says "Experiments indicate that the sweet spot here is around ...
- 101
2
votes
2
answers
188
views
Classifier can predict time series 1 day in advance, but not more. Why?
To ask the question more precisely: when doing Time Series classification, I observe the classifier prediction is good if test data directly follows (in chronology) the train data. But when the train ...
- 71
0
votes
0
answers
738
views
What's a range of good F1 scores?
I have watched a lot of videos on machine learning and in terms of F1 scores, all are different. One video says that an F1 score of .8 is bad, but another says an F1 score of .4 is excellent. What's ...
1
vote
1
answer
1k
views
Should I balance the classifier train/test set, if metrics is Precision/Recall (F1 score)?
I want to train a classifier on an unbalanced data set. Proportions of classes C0/C1 are 65/35. Importantly, the success metrics is F1_score. In other words, the proper classification of class 1 (...
- 71
1
vote
0
answers
12
views
How to verify that one ML-classifier is better then the other using the same train and test data set without cross-validation?
I have compared 5 methods (ML classifiers) on the same data set. These methods are 5 different types of neural networks. Each is trained on the training set and evaluated with precision, recall and f1-...
- 129
0
votes
1
answer
63
views
PPV vs Sensitivity, they look the same!
I am looking at the equation PPV and Sensitivity
and I got this
PPV = TP / (TF+FN)
and
Sensitivity = TP / (TF+FN)
Which ...
- 251
2
votes
1
answer
434
views
F1 score macro-average
In this question I'll differentiate by using lower-case for class-wise scores, e.g. prec, rec, f1, which would be vectors, and the aggregate macro-average Prec, Rec, F1. My formulae below are written ...
- 220
1
vote
0
answers
1k
views
Maximize F1 Score for an imbalanced data and multi-class classification
I'm dealing with an multiclass classification problem. The data is textual and too imbalanced. I see that the models that i'm building using the character level or word level grams are always giving ...
- 21
1
vote
0
answers
778
views
Issue with training on classification metrics other than accuracy (using R and caret)
I have a binary classification problem with two classes 0 and 1. For training an XGBoost classification model, I apply a balanced data set (50% 0's, 50% 1's).
In reality, 1's are much more abundant ...
- 43
1
vote
1
answer
29
views
what is the correct interpretation of precision, recall and F1 in R?
Im using R and i had some cases of NAs for F1 when there is NA for precision and 0 for recall and also when both are 0, i also noticed that with both 0 i had f1 as Nan. So im not sure how to interpret ...
- 73
0
votes
0
answers
14
views
Why would my additional information harm my prediction score but improve ROC and F-1?
I'm trying to predict the primary crime type on a given location using the Chicago crime dataset.
Stripping out all the provided features to just:
Location Description Encoded (The location ...
- 111
4
votes
1
answer
790
views
Use F1 or maximum F1 for model comparisons?
I am comparing a ML classifier to a bunch of other benchmark F1 classifiers by F1 scores. By AUPRC, my classifier does worse than other benchmark methods. When I compared F1 score, however, I got a ...
- 91
1
vote
1
answer
2k
views
Can F1 score be equal to zero?
As it is mentioned in F1 score Wikipedia that 'F1 score reaches its best value at 1 (perfect precision and recall) and worst at 0'.
What is the worst condition that was mentioned?
- 113