I'm working on a multi-class multi-label classification problem where text (let's say comments on a website) should be assigned (possibly multiple) labels. There is a neutral (negative) class and there are 7 positive classes (e.g., aggressive, racist, sexually inappropriate, etc.). Also, as you would expect, the distribution of the data is extremely skewed towards the neutral class, i.e., there are very few inappropriate comments.
I'm trying to decide on a metric to be optimized. So, far I have decided that micro-averaged label-based metrics and example-based ones (e.g., Hamming loss) are not suitable as they give equal weight to all classes and tend to get overwhelmed by the model's performance on the majority class, whereas I care more about performance on minority classes (a good review here).
I believe that macro-F1 should be a sensible metric to optimize. However, a colleague has proposed macro-recall. I'm wondering if it would make any sense to try to optimize macro-recall in a multilabel setting.
So, here is my question:
Am I right in thinking that if Recall is to be optimized, one can hack the metric and achieve a perfect macro-averaged recall by predicting all the labels for all the test cases (assuming labels are not mutually exclusive)?
Put differently, wouldn't optimizing Recall induce the model to be overly aggressive in prediction and predict as many labels as it can for test cases.