I am dealing with a classification problem with a dataset containing 60k rows: 69k are negative class, and 1k is positive. I trained my models and I obtained the confusion matrices with a threshold of 0.5 (by default). If I decrease the threshold results improve. I found that the best results are with threshold 0. But what does it mean threshold 0, is it useful? Or is it better also a threshold of 0.01 with slightly worse?
EDIT:
You can plot the ROC curve and the point that makes max the TPR and min the FPR. It does not requiere too much code, but I don't know if code is allowed in this channel. Another solution could be to rebalance the target distribution. There are some methods to achieve this. Take a look at SMOTE
Talking about threshold = 0, as Dave and Mark has said, it would mean that your model would classify EVERY observation into one group, totally ignoring the other one. IMO, this would make the model senseless, since you don't need a model to "predict" always the same value, so I would avoid threshold = 0 (or = 1)
For a highly unbalanced data set like yours, a model that always predicts 1 (as Dave was discussing) will always have high values for the metrics you're displaying. You need to focus on the negative predictions; if the interface you're using allows the selection of the metric true negative rate (also called specificity or selectivity) use that. If such an option is not available, see if you can switch from identifying 1 as the positive label to identifying 1 as the negative label; I think that would put the focus on the negative labels where it belongs.
I had come cross similar imbalanced data recently and by using F1 score metrics, I picked up 0.09 as best classification threshold. So as other suggests, you should use other metrics (F1 score in scikit) to decide best threshold.
I also observed SMOTE provides reasonable results for imbalanced data.
