I would like to generate a confidence interval for predicted vs actual rates.
I am auditing my group of anaesthetists (aka anesthesiologists) to see how we compare on a number of potentially preventable complications (eg post-operative nausea, severe pain, hypothermia).
I have 20000 surgical operation records and I can make a GLM to make a "case-mix adjusted risk" (using age, gender, type of surgery, duration of surgery as risk factors) and thus generate a risk for each patient.
I can then aggregate the risk and actual per clinician I can generate an actual and predicted rate for each complication. I can make a confidence interval for my actual - but it seems a bit simplistic to just test to see if the confidence interval on the actual rate includes the rate generated from summing the glm-predicted risks.
This question has some pointers to a package but I am hoping for some more specific suggestions.
To clarify what I have already (using "requirement for pain protocol" as an example):
# make model (dependent variable has values 1/0) model.pp = glm( pain_protocol1 ~ age + log_age + age2 + inv_age + op_time + log_op_time + op_time2 + gender + category + thimble, family = "binomial", data=d4) # calculate predicted PACU time and then difference between predicted and actual: d4$pred_pp = predict(model.pp, newdata=d4, type="response", na.action="na.pass") d4$extra_pp = d4$pain_protocol1 - d4$pred_pp # aggregate deviation from predicted rate ppr_pa <- aggregate(extra_pp ~ adult_anaesthetist, data=d4, FUN=mean) barplot(ppr_pa$extra_pp, name=ppr_pa$adult_anaesthetist,las=2)
So I can make this plot for my colleagues, showing the variation we have in how much pain our patients experience in the "post anaesthesia care unit". These variations are great enough so that they definitely represent a material difference in patient experience, and most of the difference will also be unlikely to be variation due to chance (ie "statistically significant"). However, as I examine smaller subgroups and other complications that are less frequent it would be good to be able to calculate confidence intervals.
Note that each clinician has a different number of cases, and each clinician is given a bird code-name for anonymity.