I have a binary classifier with 94% accuracy on unknown test data. I use that model to classify samples from a large population in order to infer the proportion of positives within the population. I use Clopper-Pearson to construct a confidence interval with a significance level of $\alpha=0.05$, so that the true proportion of positives is contained in my predicted interval 95% of the time.
Now, given my classifier's accuracy of 94%, 4.2% of false negatives and 1.6% of false positives, should I adjust my confidence interval in some way? Intuitively, it makes sense to widen the interval, because of the added uncertainty of my classifier. How would I go about that, while keeping everything statistically correct?
Or how does the accuracy affect the significance level?