Calculate the confidence interval of a balanced accuracy by taking the mean of the CIs of sensitivity and specificity? Because sensitivity and specificity are typically estimated as binomial proportions (e.g. k = TP, n = TP+FN), we can use any of the methods used to estimate the confidence interval for binomial distributions to quickly calculate the CIs without using bootstrapping.
The CI of accuracy can also be quickly calculated using the same method by picking the values from the confusion matrix of binary classifier (i.e. k = TP+NP, n = N). However, this is not possible for balanced accuracy, which gives equal weight to sensitivity and specificity and can therefore not directly rely on the numbers of the confusion matrix, which are biased by prevalence (like accuracy). The formula for balanced accuracy is
$$
BACC = \frac {Sensitivity + Specificity}{2}
$$
Hence, my thought is to simply use this formula for the lower and upper bounds of the CI. That is,
$$
\text{lower bound of BACC CI} = \frac {\text{lower bound of Sensitivity CI} + \text{lower bound of Specificity CI}}{2}
$$
$$
\text{higher bound of BACC CI} = \frac {\text{higher bound of Sensitivity CI} + \text{higher bound of Specificity CI}}{2}
$$
It makes a lot of intuitive sense and the values seem to make sense. However, I wondered if this is actually a sensible and sound method to calculate the CI of balanced accuracy.
 A: While I'm not at all convinced balanced accuracy is a useful summary, that's also not how you compute a confidence interval for it.
To a reasonable approximation, the estimated sensitivity and specificity will be Normally distributed around the true values.
If
$$\widehat{\mathrm{sens}}\sim N(\mathrm{sens}, \sigma^2)$$
and
$$\widehat{\mathrm{spec}}\sim N(\mathrm{spec}, \tau^2)$$
then for balanced accuracy
$$\widehat{\mathrm{bla}}\sim N\left(\mathrm{bla}, \frac{\sigma^2+\tau^2}{4}\right)$$
You can compute $\sigma$ and $\tau$ by dividing the confidence interval lengths for sensitivity and specificity by $2\times 1.96$
A: I have been looking into this a bit more, and it seems as though a Normal confidence interval plus a logit transformation does very well in modest sample sizes.
As earlier, define
$$\widehat{\mathrm{sens}}\sim N(\mathrm{sens}, \sigma^2)$$
and
$$\widehat{\mathrm{spec}}\sim N(\mathrm{spec}, \tau^2)$$
then for balanced accuracy
$$\widehat{\mathrm{bla}}\sim N\left(\mathrm{bla}, \frac{\sigma^2+\tau^2}{4}\right)$$
Now take a logit transformation
$$\mathrm{logit}(\widehat{\mathrm{bla}})\sim N\left(\mathrm{bla}, \frac{\sigma^2+\tau^2}{4\mathrm{bla}^2(1-\mathrm{bla})^2}\right)$$
compute a confidence interval $(l,\,u)$ for $\mathrm{logit}({\mathrm{bla}})$ using this Normal approximation, then transform back to the probability scale as
$(\mathrm{expit}(l),\,\mathrm{expit}(u))$
A: Sensitivity and specificity are two entirely different measures of the usefulness of a test. One is based on a (presumably small) population of subjects who have the disease or condition; the other is based on a (presumably much larger) population of subjects who don't.
I can see no valid rationale for averaging the two. As an example, suppose a test has sensitivity 99% but its specificity is 1%, essentially rendering the test useless.

*

*A bogus test that just declares 99% of subjects to be 'positive', absent all contact with reality, would do as well.


*Then how could you justify a definition of 'test accuracy' to say the test is  "50% accurate"?
Example: Consider a population of 100,000 with 5% prevalence so
that 5000 have the disease and 95,000 do not. Especially in
developmental stages it is not unrealistic for a test to have
95% sensitivity and 80% specificity.
Here are the consequences of testing everyone in the population:

*

*4900 correctly treated or quarantined due to true positive results, and
100 undetected potential 'spreaders' of the disease.

*19,000 incorrectly quarantined or treated (by whatever means) due to false positive results,
and 76,000 with no direct consequences of testing.

Especially considering that any member of the population can get the disease at any point in time, the situation is sufficiently difficult that unjustified simplifications are not likely to be helpful.
