2
$\begingroup$

I am assessing the performance of a hospital ward over 6 years. There have been two sets of changes to the ward over this time: after year 2 and after year 4.

I have collected data in 4 sample months for each of the three time periods (baseline, following change 1, and following change 2). The denominator data is the total number of admissions to the ward in each month. I am trying to compare mortality rates, 30-day readmission rates, etc. (12 data points for each variable).

Can the proportions generated by these rates (dependent variable/total admissions that month) be compared by one-way ANOVA for each dependent variable, or do I have to do a square root & arcsine transformation first? Each of the dependent variables is discrete, but probably distributed approximately normally.

Am I missing the point entirely? I.e. can I even use ANOVA, or do I need to use Fisher's exact test, as the data is technically discrete and converting it to a proportion does not make it continuous? Thanks.

$\endgroup$
3
  • 1
    $\begingroup$ "Each of the dependant variables are discrete but are probably distributed normally." is contradictory. You may mean 'approximately normally', but you can't mean 'normally'. The discreteness might (or might not) be less important than the issues of independence and constant variance. $\endgroup$
    – Glen_b
    Feb 6, 2014 at 23:49
  • 1
    $\begingroup$ Yes, apologies, each of the variables would approximate normally. They are not independent of the denominator i.e. one would expect more deaths in a month with more admissions $\endgroup$
    – romsdocs
    Feb 7, 2014 at 0:09
  • 1
    $\begingroup$ You could treat number of admissions as an exposure variable in a GLM $\endgroup$
    – Glen_b
    Feb 7, 2014 at 0:51

1 Answer 1

3
$\begingroup$

You raise some interesting issues with your question. I agree with the comments by @Glen_b above and I would like to expand on them.

It sounds like your response variables are in the form of success/trials, making them proportions. ANOVA-type models based on normal distributions are often used with proportional data, with varying success. Technically, proportional data are not normal—they are bounded on [0,1], they can only take on values that are multiples of 1/trials, and there is no guarantee that they are symmetric (in fact, they are often skewed). That being said, normal-based analyses of proportional data may still yield a “correct” conclusion.

If you consider each monthly admission to be an independent Bernoulli process with only two possible outcomes (i.e. success = die and failure = live for mortality rate; success = readmission and failure = no readmission for readmission rate, etc.) then you can assume that the monthly response variables are distributed as binomial(n,p) . It’s well-accepted in statistical literature that the normal distribution approximates the binomial distribution effectively when the binomial proportions are between about 0.3 and 0.7. If your proportions are in this range then normal-based models might work. However, non-constant variance and/or lack of independence could still be major problems.

You are correct that one option might be to use the arcsine square root transformation to make the proportions more normal with stable variance and the F tests from normal-based models more valid. But this approach is antiquated and undesirable for many reasons. ANOVA-type models are not limited to normal distributions. A better option is to use generalized linear model (GLM) methods to model the data with a binomial distribution and logit link. Because of overdispersion you may find that a beta distribution, which is defined on [0,1] and so is a logical choice for proportional data, works better. Or you may wish to add a random experimental unit effect to the model.

This brings up another point—it’s not clear to me what the actual experimental unit for your “study” really is. (An experimental unit is the entity to which a treatment is applied independently.) If the three “treatments” (baseline, change1, change2) were applied at the level of the ward then you really only have one experimental unit. Even though your data set has many numbers in it, you cannot escape the fact that you have an effective sample size of 1. Each month and admission is really just a subsample. If this is the case then any ANOVA approach—normal or non-normal—is inappropriate, and so is Fisher’s exact test for contingency table analysis. Without having additional hospital wards as replicates, you will have to compare summary statistics directly without the aid of statistical tests.

But if the treatments are applied at the level of each individual admission then these tests might be appropriate. However, there are still some problems. The most obvious one is with independence. The same patient can be part of multiple admissions in the same month, in multiple months, and across multiple treatments. Your statistical model needs to account for this. Also, your choice of allocating the response variables into monthly intervals can profoundly affect your model results. What happens if a patient is admitted in a month that you sampled but is readmitted and/or dies in a month that you did not sample? It is possible that the monthly intervals create a false notion of replication and also serve to censor data.

It is dangerous to use a one-way ANOVA here because you are comparing the same ward and this affects your inference space. A better approach would be to use a repeated measures-type model and construct contrasts to assess treatment differences. Also, you don’t have a real control. Even though your “baseline” period acts like a control, there are many confounding effects that your study does not address. For example, changes in performance that were actually caused by turnover in staff or by factors external to the ward are confounded with “change 1” and “change 2.” Your design also does not address carryover effects that operate on scales greater than 2 years.

$\endgroup$
3
  • $\begingroup$ The problems you have highlighted with the model relate to these sorts of studies generally but I am trying to evaluate the effectiveness of major organisational changes which occurred at distinct times (i.e. overnight overhaul of processes). With regards to your points on and mortality and readmissions, I looked at patients admitted in the sample months, and if they were readmitted within 30 days of their discharge date or died in hospital during their admission (whenever that was - even upto 145 days!). Admission >30 days post discharge is not classified as readmission. $\endgroup$
    – romsdocs
    Feb 9, 2014 at 17:23
  • $\begingroup$ Would parametric bootstrapping be another possible method of analysis of the proportions as it does away with concerns about how the data is distributed? See here (bmj.com/content/320/7243/…) $\endgroup$
    – romsdocs
    Feb 9, 2014 at 17:25
  • $\begingroup$ You certainly can try parametric bootstrapping, but keep in mind that violating independence is generally far more consequential than violating distributional assumptions. Recall, for example, that the Gauss-Markov theorem does not require normality. You may get similar results with multiple models (parametric and nonparametric) but GIGO still applies and your results may not be valid. $\endgroup$ Feb 10, 2014 at 21:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.