I was going through many state-of-the-art papers in Meta-learning and few-shot learning, and I found that almost all use "accuracy" as evaluation criterion. Unlike other domains like object detection, NLP and other machine learning domain which use precision, recall, MAP, ROC etc. apart from accuracy. Why meta-learning uses only accuracy and no other parameter?
-
$\begingroup$ Why is accuracy not the best measure for assessing classification models? The same problems also apply to precision, recall etc. AUC is slightly better: stats.stackexchange.com/q/339919/1352 $\endgroup$– Stephan KolassaNov 2, 2021 at 5:48
1 Answer
One reason is coming from the dataset structure: Consider the few-shot learning, suppose we are trying to identify a person from one photo in testing/production stage, and in training stage, we only have few photos for each person.
The natural question can be, if the model says the photo is person A, how accurate it can be (such metric can be extended into top N accuracy instead of top 1 accuracy).
Note that, the concept of the "precision" and "recall" cannot be easily applied there.
Because in testing stage, we only have one photo.
For precision, we are checking the "false positive rate", we need to have more test instances tested for a given person and check the how much false positives are there.
For recall, it means how many "true positive" we can find in a large dataset (assume we have a lot of positives). For few-shot learning use case, this is also not applicable: where we only have very few true positives (very few instances for each class).