Establish the proper prediction threshold

Question

Suppose for a ML classification problem X, I have three labels, i.e. 1, 0 and -1. For some reasons, I obtained bad accuracy. So I decided to change the number of labels to two, i.e. 1 and -1. That way I got much better result. Now, when I am in the prediction phase, I want first to find what is my threshold level for precision/recall. If the prediction is not within those levels, assume 0. Programmatically, what is the proper way to establish that threshold?

Answer 1

Do not use classification thresholds unless you understand the trade-offs of your decision.

Do not use accuracy to evaluate a classifier: Why is accuracy not the best measure for assessing classification models?

(See here for a general motivation for this answer. See here for a motivation for short answers. Longer answers are always welcome.)