Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I recorded hand hygiene type for 500 nurses following 500 episodes of care. I now have a probability for each type of hygiene method: Alcohol gel, Soap, Gloves, None.
But how do I calculate a confidence internal for these? Is that the best way to report the findings? Any help would be much appreciated.
Let's assume that all nurses and episodes are basically the same. Let $X_i$ be the choice of hygiene type for a particular episode/nurse. To estimate the probability of $X_i$ we usually use a count (which is MLE):
$$ p_j=E[I(X_i=j)] $$
Where $I$ is an indicator function. The variance of an indicator function like this is just:
$$ \text{Var}(I(X_i=j))=E[I(X_i=j)^2] - E[I(X_i=j)]^2 = p_j(1-p_j) $$
The sample equivalent of the count above is:
$$ \hat p_j=\dfrac{1}{n}\sum_{i=1}^nI(X_i=j) $$
The CLT tells us that:
$$ \sqrt{n}(\hat p_j - p_j)\xrightarrow{d}\ N\left[0,\;\text{Var}(I(X_i=j))\right]. $$
by the CLT, a consistent estimator of the variance of $\hat p_j$ is then:
$$ \dfrac{1}{n}\hat p_j (1-\hat p_j) $$
so the confidence interval for $\hat p_j$ can be written as: $$ \hat p_j \pm z_q \sqrt{\dfrac{1}{n}\hat p_j (1-\hat p_j)} $$
where $z_q$ is the $1-q/2$ quantile of the standard normal distribution (1.96 for $q=0.05$).
This asymptotic approximation works well when $\hat p_j n>5$ and when $\hat p_j\neq 0, 1$. Otherwise you will have to use one of the alternatives here. I believe Clopper-Pearson is particularly popular. Let $c(X) = \sum_{i=1}^nI(X_i=j)$. Then the Clopper-Pearson upper bound and lower bound of the confidence interval are:
\begin{align} \text{Upper bound} &= 1-B^{-1}\left(\frac{q}{2}, n - c(X), c(X)+1\right)\\ \text{Lower bound} &= 1-B^{-1}\left(1-\frac{q}{2}, n - c(X) +1,c(X)\right) \end{align}
where $B^{-1}$ is the inverse of the Beta distribution. This confidence interval follows directly from the binomial distribution, instead of working through asymptotic normality.
If you want to calculate probabilities for each nurse, you can apply the same formulas on a nurse-by-nurse basis, subject to the same caveats. For more complicated models, each should have it's own asymptotic variance attached.
