1
$\begingroup$

Suppose I have two binary classifiers, A and B. Both are trained on the same set of data, and produce predictions on a different (but same for both classifiers) set of data. The precision for A is high and the recall is low, whereas the precision for B is low and the recall is high. Is there a way to combine these two models where I can get the precision from one and recall from the other, or possibly use the metrics from one to improve those of the other?

For example, let's say these are the metrics:

A

Precision: 0.91

Recall: 0.35

B

Precision: 0.43

Recall: 0.90

As these are binary classifiers, my labels are 1 and 0, and my class of interesting is 1 (so the metrics above are for predicting 1s).

Let's say model A predicts 10 1s, and model B predicts 70 1s.

It is safe for me to say that of the 10 1s that model A has predicted, 9 are true positives.

It is also safe for me to say that of the 70 1s that model B has predicted, 60% are false positives, but the rest are 90% of the true positives in the dataset.

My question is, is there some method for me to combine these results so that I can obtain all 50 true positives from the dataset?

$\endgroup$
  • $\begingroup$ Can you be more specific? $\endgroup$ – Michael Chernick Mar 25 at 18:04
  • $\begingroup$ Edited to try to make it clearer. Does this help? $\endgroup$ – user8115948 Mar 25 at 18:15
0
$\begingroup$

I think when you classifier A has a high probability you can trust classifier A. Now the problem is that A misses a LOT and B predicts a LOT. I can think that since both are trained on the same set of data you can try playing with the classification threshold. This will help you better tune Precision & Recall for a model

Also, I believe you can build a model on top of the predictions of model A & B. Use the predicted probabilities and labels as features and try to predict the correct class either - using logistic regression will be good.

$\endgroup$
  • $\begingroup$ I can't play with the threshold as it is already 0.5. Could you explain a bit more about what you mean by the second paragraph? I would just like to know how to use the above metrics to maximize the amount of true positives I can get $\endgroup$ – user8115948 Mar 27 at 14:13
  • $\begingroup$ Most classifiers have a way to output probabilities. I would like to know what kind of classifier you are using. 2nd paragraph - You build a model where the output is your true label and it's features are the predicted probabilities of model A and of model B. I also think that usually in problems precision and recall have different costs, hence we can optimize the hyperparameters/threshold to optimize on one. $\endgroup$ – Axelius Mar 27 at 14:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.