I have a logistic regression recommender model built on my data where I tried to predict one of two outcomes for each row. Let's call them
fail. I'm using the
cross_val_score function of
SKLearn to do so. I was planning on just using precision and recall as performance measures, but saw the "micro" and "macro" options and decided to read up more and try those as well, but I'm confused by the results. For reference, I looked at this blog post, this SO question, and this paper.
While I think I understand the general concept, I'm confused by the results I'm getting, and which one is ideal for my use case. One important piece of background is that I have fewer
success cases that
success makes up 25% of my dataset.
My understanding is that micro is a value that's closer to the performance of the model on the larger class (in this case,
fail), while macro is for the smaller one (
success). This makes sense. I'm trying to predict successes in this case, so the latter should be the better choice.
Here are the parts I don't get:
a) How do each of these compare to just regular precision / recall, in terms of representing the larger or smaller classes? Which of the three options is optimal for a case like this where I want to surface the smaller class but not the large one.
b) In some folds of my CV, I get
0 as the regular precision value, but none are
0 for micro and macro. How is that possible? If I get some true positives in the individual classes, shouldn't I always have some in the overall dataset as well? In general the regular precision and recall end up significantly lower than both types of averages.