# Calculating the standard error of sensitivity and specificity

For a method, I'm calculating it's sensitivity and specificity. I also want to calculate standard errors, but I'm unsure how. I don't have a sample to calculate it from. All I have are 50 iterations where I applied the method to a different dataset each time. So that, I have 50 values for the number of TPs, FPs, TNs, FNs. From this, I calculate the sensitivity and specificity by summing all TPs, FPs, TNs, FNs.

I haven't been able to find much information online, aside from a wikihow article, which states that it can be calculated as:

$$\sqrt{\frac{(1-Sensitivity)*Sensitivity}{n_p}} * 1.96,$$

where $$n_p$$ is the number of positives in the dataset. Is this correct?

Let's define some notation.

$$y\in\{-1,+1\}$$ is the true state.

$$\hat y\in\{-1,+1\}$$ is the predicted state.

Then...

$$\text{Sensitivity} = \mathbb{P}\big(\hat y = +1 \vert y = +1)\\ \text{Specificity} = \mathbb{P}\big(\hat y = -1 \vert y = -1)$$

So both of these are just proportions. Treat them like you would any other proportion parameter of a Bernoulli distribution. That's where you get the standard error formula you gave. The common way to calculate standard error (SD) is the following: The outcome of each experiment is Bernoulli distributed with parameter $$p$$ $$X_1,\dots,X_n \overset{iid}{\sim} Bern(p)$$. The number of successes is then binomially distributed (since order does not matter) and we need to deal with the https://en.m.wikipedia.org/wiki/Binomial_proportion_confidence_interval The standard error for the proportion is then $$SD(\hat{p})=\sqrt{\dfrac{\hat{p}(1-\hat{p})}{n}}$$. Other possible ways to compute a confidence interval are highlighted in Wikipedia.

(I dispute that the $$1.96$$ should be included. Standard errors should not be tied to particular confidence levels. What if I want a $$90\%$$ confidence interval or a $$99\%$$ confidence interval?))

To get from there to your formula, the $$\hat{p}$$ is the sensitivity, and the $$n$$ is the number of positive cases, which you are calling $$n_p$$. Likewise, the standard error for specificity would use $$\hat{p} = \text{specificity}$$ and $$n$$ equal to the number of true negative cases.

• I don’t see a nice way to make the prose flow, but $\hat p$ is the estimate of $p$, calculated as the proportion of $1$s in the data.
– Dave
Aug 28, 2021 at 4:39