How do we calculate a confidence interval for a result in binary classifiers ?
CI for regression problems makes sense since we have a variable estimated output that I can calculate its estimated mean and then get the SE around it.
For classification problems, We only have metrics like Fpr/Tpr/AuC, precision/accuracy & class probabilities. Besides, class distribution is not usually approximated to a known distribution.
I am implementing a RandomForest classifier via Python for a biased binary classification problem.