You raise some interesting issues with your question. I agree with the comments by @Glen_b above and I would like to expand on them.
It sounds like your response variables are in the form of success/trials, making them proportions. ANOVA-type models based on normal distributions are often used with proportional data, with varying success. Technically, proportional data are not normal—they are bounded on [0,1], they can only take on values that are multiples of 1/trials, and there is no guarantee that they are symmetric (in fact, they are often skewed). That being said, normal-based analyses of proportional data may still yield a “correct” conclusion.
If you consider each monthly admission to be an independent Bernoulli process with only two possible outcomes (i.e. success = die and failure = live for mortality rate; success = readmission and failure = no readmission for readmission rate, etc.) then you can assume that the monthly response variables are distributed as binomial(n,p) . It’s well-accepted in statistical literature that the normal distribution approximates the binomial distribution effectively when the binomial proportions are between about 0.3 and 0.7. If your proportions are in this range then normal-based models might work. However, non-constant variance and/or lack of independence could still be major problems.
You are correct that one option might be to use the arcsine square root transformation to make the proportions more normal with stable variance and the F tests from normal-based models more valid. But this approach is antiquated and undesirable for many reasons. ANOVA-type models are not limited to normal distributions. A better option is to use generalized linear model (GLM) methods to model the data with a binomial distribution and logit link. Because of overdispersion you may find that a beta distribution, which is defined on [0,1] and so is a logical choice for proportional data, works better. Or you may wish to add a random experimental unit effect to the model.
This brings up another point—it’s not clear to me what the actual experimental unit for your “study” really is. (An experimental unit is the entity to which a treatment is applied independently.) If the three “treatments” (baseline, change1, change2) were applied at the level of the ward then you really only have one experimental unit. Even though your data set has many numbers in it, you cannot escape the fact that you have an effective sample size of 1. Each month and admission is really just a subsample. If this is the case then any ANOVA approach—normal or non-normal—is inappropriate, and so is Fisher’s exact test for contingency table analysis. Without having additional hospital wards as replicates, you will have to compare summary statistics directly without the aid of statistical tests.
But if the treatments are applied at the level of each individual admission then these tests might be appropriate. However, there are still some problems. The most obvious one is with independence. The same patient can be part of multiple admissions in the same month, in multiple months, and across multiple treatments. Your statistical model needs to account for this. Also, your choice of allocating the response variables into monthly intervals can profoundly affect your model results. What happens if a patient is admitted in a month that you sampled but is readmitted and/or dies in a month that you did not sample? It is possible that the monthly intervals create a false notion of replication and also serve to censor data.
It is dangerous to use a one-way ANOVA here because you are comparing the same ward and this affects your inference space. A better approach would be to use a repeated measures-type model and construct contrasts to assess treatment differences. Also, you don’t have a real control. Even though your “baseline” period acts like a control, there are many confounding effects that your study does not address. For example, changes in performance that were actually caused by turnover in staff or by factors external to the ward are confounded with “change 1” and “change 2.” Your design also does not address carryover effects that operate on scales greater than 2 years.
