2
$\begingroup$

This might be a dumb question but I'm doing a basic data analysis for a medical group. Previously, I did a project for them where we looked at patient outcomes in a major hospital (let's call it "Hospital A") to evaluate doctor performance and at Hospital A, we had covid testing data per patient and were able to break it out to see how different types of patients performed. Covid patients obviously did far worse than historical algorithms predicted while non-covid patients actually did much better, possibly due to lifestyle changes that occurred during the pandemic, maybe.

Anyways, they're hiring me again to do the same thing for another hospital that's nearby ("Hospital B"). The only issue is that it uses a different database that doesn't have covid testing by individual. All they have is the total number of covid vs non-covid cases by year.

For reference:

Hospital A had a 12% covid rate

Hospital B had a 9% covid rate

Before covid, they would just use a program called APACHE (not the programming software) to generate a predicted mortality rate per patient and to evaluate the doctor's performance, they would use the risk-adjusted mortality rate which is simply equal to observed mortality/predicted mortality. This algorithm wasn't tuned to a once-in-a-century virus obviously so that's the reason why they need a data nerd like me to unpack it and make it more granular.

If in Hospital A, we found that the predicted rate was off by 22%, would it be acceptable from a statistics standpoint to simply do:

corrected_predicted_mortality_rate = initial_predicted_mortality_rate * (9%/12%) * 22%

$\endgroup$

1 Answer 1

1
$\begingroup$

Your formula works if there are no interactions. For instance, that every type of patient has equal chance of getting COVID and that for every type of patient there is a same effect of COVID. In addition you need to assume that only the COVID patients change in rate and not the non-COVID patients.

Your formula works in the following example

Let's for simplicity assume that the change in mortality rate was 20% in hospital A (such that divisions become easier) and that there are 10% COVID patients in hospital A and 20% COVID patients in hospital B

Let hospital A have 10 patients with COVID and 90 patients without COVID. Before they had 10% mortality rate. So that is 1 for each 10 patients. Now COVID doubles the probability of death and hospital A will be having 2 patients dying out of the 10 COVID patients and 9 patients dying out of the 90 non-COVID totalling 11 patients which is a 10% increase.

Let hospital B have 20 patients with COVID and 80 patients with COVID. Before they had 20% mortality rate, so 2 out of every 10 patients. If we assume that COVID doubles the mortality rate similar to hospital A then now we will have 8 dying patients for the 20 with COVID and 16 for the 80 without COVID making a total of 24% which is a 20% increase.

This is similar to your formula but you have to use

$$\text{increase % hospital B}= \text{increase % hospital A} \cdot \frac{\text{% COVID patients hospital B}}{\text{% COVID patients hospital A}}$$

Your formula is

$$\text{corrected_predicted_mortality_rate} = \text{initial_predicted_mortality_rate} * (9\%/12\%) * 22\%$$

or

$$\frac{\text{corrected_predicted_mortality_rate}}{\text{initial_predicted_mortality_rate}} = (9\%/12\%) * 22\%$$

But should be

$$\text{percent change mortality hospital} = (9\%/12\%) * 22\%$$

Counterexample

But now let's also change the mortality among non-Covid.

Again, let's for simplicity assume that the change in mortality rate of hospital A was 20% (such that divisions become easier) and that there are 10% COVID patients in hospital A and 20% COVID patients in hospital B.

Let hospital A have 10 patients with COVID and 90 patients without COVID. Before they had 10% mortality rate. So that is 1 for each 10 patients. Now COVID triples the probability of death and hospital A will be having 3 patients dying out of the 10 COVID patients. But the mortality decreases by 1/9-th and now 8 patients are dying out of the 90 non-COVID totalling 11 patients which is a 10% increase.

Let hospital B have 20 patients with COVID and 80 patients with COVID. Before they had 20% mortality rate, so 2 out of every 10 patients.

Let's do the computation with the triple and 1/9-th changes

12 patients die in the COVID group and 16*(8/9) die in the non-COVID group. This totals about 26.22 patients and that is a 31.1% increase.

So now the formula does not work. Before hospital B had a 20% increase and now it is a 31.1% increase.

$\endgroup$
2
  • $\begingroup$ The description in the answer here is related to the Yule-Simpson effect. You can 'solve' it by looking better at the type of patients and include demographic factors like age and factors like disease type and patient status. But that just relates to what type of formula to use. You still have to consider the statistical issue of variations that occur and you need to describe the uncertainty due to the sampling. $\endgroup$ Commented Apr 6, 2022 at 22:48
  • $\begingroup$ This is a beautiful response and thank you very much for adding all the detail explaining the reasoning and what assumptions are being made. $\endgroup$
    – smoalem
    Commented Apr 8, 2022 at 4:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.