I am testing a binary classifier on a highly negative dataset and recently learned of the Matthews Correlation Coefficient (MCC). I had been using positive predictive value (precision) and true positive rate (recall) to tune my classifier, but found that precision is negatively influenced as the number of negative samples go up, so I am hoping MCC can be a better guide.

However, mine is a text retrieval application where I'm more interested in recall than precision. I would like to use something like the F2 score, but again, F2 is influenced by the number of negative samples.

Is there a way to adjust MCC to favor recall much in the same way that F2 favors recall over F1? Would I apply similar biased harmonic mean multipliers as was done to produce $F_\beta$ scoring?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.