Questions tagged [f1]
The f1 tag has no usage guidance.
45
questions
1
vote
1answer
11 views
How to calculate F1, Precision, and Recall for Multi-Label Multi-Classification
I have a predictive model as follows
Sample1
Sample2
Sample3
Sample4
Red
Yellow
Blue
Green
White
Black
Orange
65
21
55
40
0
0
1
0
1
0
0
31
40
44
30
0
0
0
0
0
0
0
33
44
56
66
1
0
0
1
0
0
1
63
77
...
0
votes
1answer
35 views
Model accuracy versus F1
When training a model (classifier) in TensorFlow, an accuracy value is returned. What is the interpretation of an accuracy of, say, ...
0
votes
0answers
11 views
Is there a F1-score for the negative class?
I am analysing some data and calculated the $F1 = \frac{2TP}{2TP + FP + FN}$
I'd like to know if there is a name to a negative-class based F-score, something like $F0 = \frac{2TN}{2TN + FP + FN} = 2 \...
0
votes
0answers
24 views
How to calculate chance-level f1, ROC-AUC, PR-AUC for imbalanced dataset
I have an imbalance dataset (60% class 1, 40% class 0).
I trained a model and got accuracy, f1, ROC-AUC and PR-AUC.
I want to compare them to chance-level performance.
obviously chance-level of acc if ...
0
votes
2answers
24 views
What is the F1 Score for my prediction when all values are negative?
I have built a model that gives me classification of some cases
here is a comparison between Actual and Prediction
...
0
votes
0answers
8 views
How to evaluate F1 performances using sklearn for imbalanced multiclass classifier, with different weighting for train and test sets?
I have an imbalanced dataset. The number of occurrences\weight of each class in the test and train sets is different. I wish to use the sklearn implementation of the F1 score for the evaluation. I am ...
1
vote
1answer
24 views
F1 Score is giving good value in imbalanced dataset
If I have an imbalanced dataset that consists of 90% positive points and 10% negative points. Now I created a "dumb" model which always predicts every point as a positive point. The ...
0
votes
0answers
18 views
Continuously occurring true negatives
How can I handle discrete events in a continuous time stream in the context of an F1 metric?
To give an example, let's say the Earthquake Forecasting Bureau would report the following for their ...
0
votes
1answer
77 views
Question answering bot: EM>F1, does it make sense?
I am fine-tuning a Question Answering bot starting from a pre-trained model from HuggingFace repo.
The dataset I am using for the fine-tuning has a lot of empty answers. So, after the fine tuning, ...
1
vote
0answers
15 views
Average Precision vs average F1
Average precision computes the area under the recall-precision curve by the trapezoidal rule (or midpoint rule). However, we could also compute the F1 score for every threshold and then take the ...
0
votes
1answer
23 views
hyperparameter search with unknown test set distribution
I'm training a 3-class neural network classifier (conv layers and softmax at the end, nothing special). Let's say, in the test set I will have N1 examples of the 1st class, N2 examples of the 2nd ...
3
votes
1answer
80 views
Why use harmonic mean for precision and recall (f1 score) instead of just the product of precision and recall?
General question here, I understand the purpose of using the harmonic mean to generate the f1 score for model evaluation. I'm not exactly sure though why we don't just take the product of precision ...
1
vote
1answer
372 views
On which set (train/val/test) do people calculate F1 score, precision and recall?
This may be a stupid question, but when I was looking at the definition of precision/recall etc. it was not mentioned anywhere which set (training/validation/test) this metric should be calculated ...
0
votes
0answers
16 views
How to explain a relationship between Accuracy and F1 Score / F-Measure?
I am building a CNN model for pitch estimation using a song recording. For the evaluation metrics, I am using Accuracy and ...
0
votes
0answers
288 views
f1-score of imbalanced data within k fold cross validation
I am trying to find the f1 score, precision, recall of a highly imbalanced dataset. I would like to use k-fold cross validation approach. I followed the procedure:
create arrays to store testing data ...
0
votes
0answers
45 views
How to calculate f1 score for anomaly detection with one class (Anomaly or No Anomaly)? What is seen as the true positive?
How do we calculate the f1 score in anomaly detection (using a One-Class-Support Vector Machine(OC-SVM))? I am not sure what is considered as a true positive? Is it if I predict an anomaly and the ...
2
votes
1answer
32 views
Harmonic is used in F1 score because it is a conservative metric: How does it help being conservative?
I was reading Jurafsky 3rd edition, page 12-13 chapter 4
Can you explain why is it good to weigh more the smaller of the two items namely $\frac{1}{Precision}$ or $\frac{1}{Recall}$?
Here is the link ...
0
votes
1answer
104 views
Are Jaccard score and F1 score monotonically related?
I have compared the rankings obtained by comparing 10+ classifiers with this two metrics:
Jaccard score
F1 score
They show a perfect correlation. This results holds on 50+ datasets. When comparing ...
0
votes
0answers
8 views
What ratio of skewness is typically a switch from accuracy to f1
I have a dataset where the targets are binary with the ratio 2:1
What ratio of skewness is typically a switch for a classification metrics accuracy vs f1?
So far I think I know:
If I would have 1:99 ...
0
votes
0answers
401 views
Are F1 score and Dice coefficient computed in same way or different way in image segmentation (two class segmentation)?
On page 8 of the paper An automatic nuclei segmentationmethod based on deep convolutional neuralnetworks for histopathology images, the authors show performance of their deep model on test sets. They ...
1
vote
1answer
257 views
F1 weighted vs. Log loss in SciKit learn RandomSearchCV
I am sorry to ask another question regarding this topic but I am still puzzled about the following:
When I use 'F1_weighted' as my scoring argument in a ...
0
votes
0answers
7 views
Performance Metrics on Within Subjects Study
I ran a simple yes or no within-subject experiment. i.e there were 4 conditions, and each participant answered a single question for a subset of images per condition. My question is how would I ...
6
votes
1answer
425 views
F1 score, PR or ROC curve for regression
Due to my background as a pure biologist, I've been struggling with the comment acquired from a reviewer about the accuracy test used in my regression study. While I stick to MSE, MAE and R2 as the ...
0
votes
1answer
28 views
Is it possible for a binary classifier to have lower accuracy, macrof1 and binaryf1 but higher ROC AUC? [duplicate]
I've got the results of two classifiers based on 5 different splits of training and testing sets. Their mean and std of the results are as follow:
Method-------Accuracy -- MacroF1 -- BinaryF1---- ROC ...
0
votes
0answers
126 views
Imbalanced classification what is good F1 score?
For imbalanced classification (say 85:15), what is good value of F1 score? An answer https://stats.stackexchange.com/a/217343/285091 says "Experiments indicate that the sweet spot here is around ...
2
votes
2answers
171 views
Classifier can predict time series 1 day in advance, but not more. Why?
To ask the question more precisely: when doing Time Series classification, I observe the classifier prediction is good if test data directly follows (in chronology) the train data. But when the train ...
0
votes
0answers
534 views
What's a range of good F1 scores?
I have watched a lot of videos on machine learning and in terms of F1 scores, all are different. One video says that an F1 score of .8 is bad, but another says an F1 score of .4 is excellent. What's ...
1
vote
1answer
1k views
Should I balance the classifier train/test set, if metrics is Precision/Recall (F1 score)?
I want to train a classifier on an unbalanced data set. Proportions of classes C0/C1 are 65/35. Importantly, the success metrics is F1_score. In other words, the proper classification of class 1 (...
1
vote
0answers
12 views
How to verify that one ML-classifier is better then the other using the same train and test data set without cross-validation?
I have compared 5 methods (ML classifiers) on the same data set. These methods are 5 different types of neural networks. Each is trained on the training set and evaluated with precision, recall and f1-...
0
votes
1answer
47 views
PPV vs Sensitivity, they look the same!
I am looking at the equation PPV and Sensitivity
and I got this
PPV = TP / (TF+FN)
and
Sensitivity = TP / (TF+FN)
Which ...
1
vote
1answer
292 views
F1 score macro-average
In this question I'll differentiate by using lower-case for class-wise scores, e.g. prec, rec, f1, which would be vectors, and the aggregate macro-average Prec, Rec, F1. My formulae below are written ...
1
vote
0answers
762 views
Maximize F1 Score for an imbalanced data and multi-class classification
I'm dealing with an multiclass classification problem. The data is textual and too imbalanced. I see that the models that i'm building using the character level or word level grams are always giving ...
1
vote
0answers
580 views
Issue with training on classification metrics other than accuracy (using R and caret)
I have a binary classification problem with two classes 0 and 1. For training an XGBoost classification model, I apply a balanced data set (50% 0's, 50% 1's).
In reality, 1's are much more abundant ...
1
vote
1answer
23 views
what is the correct interpretation of precision, recall and F1 in R?
Im using R and i had some cases of NAs for F1 when there is NA for precision and 0 for recall and also when both are 0, i also noticed that with both 0 i had f1 as Nan. So im not sure how to interpret ...
0
votes
0answers
14 views
Why would my additional information harm my prediction score but improve ROC and F-1?
I'm trying to predict the primary crime type on a given location using the Chicago crime dataset.
Stripping out all the provided features to just:
Location Description Encoded (The location ...
4
votes
1answer
498 views
Use F1 or maximum F1 for model comparisons?
I am comparing a ML classifier to a bunch of other benchmark F1 classifiers by F1 scores. By AUPRC, my classifier does worse than other benchmark methods. When I compared F1 score, however, I got a ...
1
vote
1answer
1k views
Can F1 score be equal to zero?
As it is mentioned in F1 score Wikipedia that 'F1 score reaches its best value at 1 (perfect precision and recall) and worst at 0'.
What is the worst condition that was mentioned?
0
votes
2answers
67 views
Removing Multi-Collinearity Reduces F1 score
I was trying to build a classification model and I found that the features were highly correlated.
I tried run a random forest model on the features and got an F1 score of 0.44 but when I removed the ...
1
vote
0answers
99 views
sklearn f1_score=weighted not matching sample_weight specification
I am trying to figure out exactly what this is doing:
sklearn.metrics.f1_score(y_pred, y_test, sample_weight=[...])
Numerically it simply does not seem to be ...
0
votes
1answer
108 views
High AUC, low f1, SVM threshold for an unbalanced problem
I have a very unbalanced binary classification problem (positive class: 0.2%).
I need to evaluate it using f1 of the positive class.
Now, I'm doing some baselines using an SVM. What I get is a ...
2
votes
2answers
2k views
F1-Score in a multilabel classification paper: is macro, weighted or micro F1-used?
I read this paper on a multilabel classification task. The authors evaluate their models on F1-Score but the do not mention if this is the macro, micro or weighted F1-Score.
They only mention:
We ...
0
votes
0answers
2k views
In XGBoost with a f1_score, is the iteration with a lower or higher score the better iteration?
In the following XGBoost script the output states iteration 0 with score 0.0047 is the best score. I would expect iteration 10 with score 0.01335 to be the better score?
Output
...
2
votes
2answers
381 views
Why does the 'weighted' f1-score result in a score not between precision and recall?
On the F1 score sklearn page there's a section that explains each of the options for the average parameter. Under the weighted option, it says: "it can result in an F-score that is not between ...
4
votes
0answers
544 views
What is the difference of "normal" F1 and macro average F1 score with binary classification
Please note that I always talk about binary classification here. I do not speak about multi class classification.
In case of unbalanced binary datasets it is a good practice to use F1 score. While ...
3
votes
2answers
375 views
Reporting F1 Scores
I have a question with regard to the proper way to report F1 scores. Say I am comparing two algorithms one with F1 score of 0.71 and the other of 0.82.
Is it correct to say:
"Algorithm 1 obtained an ...
