2
$\begingroup$

I was going through many state-of-the-art papers in Meta-learning and few-shot learning, and I found that almost all use "accuracy" as evaluation criterion. Unlike other domains like object detection, NLP and other machine learning domain which use precision, recall, MAP, ROC etc. apart from accuracy. Why meta-learning uses only accuracy and no other parameter?

$\endgroup$
1

1 Answer 1

2
$\begingroup$

One reason is coming from the dataset structure: Consider the few-shot learning, suppose we are trying to identify a person from one photo in testing/production stage, and in training stage, we only have few photos for each person.

The natural question can be, if the model says the photo is person A, how accurate it can be (such metric can be extended into top N accuracy instead of top 1 accuracy).

Note that, the concept of the "precision" and "recall" cannot be easily applied there.

Because in testing stage, we only have one photo.

  • For precision, we are checking the "false positive rate", we need to have more test instances tested for a given person and check the how much false positives are there.

  • For recall, it means how many "true positive" we can find in a large dataset (assume we have a lot of positives). For few-shot learning use case, this is also not applicable: where we only have very few true positives (very few instances for each class).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.