# Testing if a proportion is different between groups

I have data on the status of a treatment in approx 800 hospitals. For example:

Hosp    N   Given   C.I Refuse  Age.Group
1       5      3     0    0      A
1       11     5     1    1      B
1       21     15    2    1      C
1       32     24    1    1      D
1       111    70    3    1      E
2       10     7     1    0      A
2       25     12    2    0      B


Each row represents one of 5 age group within a hospital. N is the number of patients with a particular condition. Given is the number of patients, out of N, receiving the treatment, C.I. is the number of patients, out of N, who were contraindicated for the treatment, and Refuse is the number of patients, out of N, who refused the treatment

I need to test whether these proportions are different between the age groups

Given / N

C.I / N

Refuse / N

Is it appropriate to do a binomial regression for each ratio, with Age.Group as the explanatory variable ?

What alternative ways are there to do this ?

-

Is your question materially focussed on the hospital element? Or are you really interested in the age group element, and it just so happens that the hospitals are providing a sampling framework?

[EDIT: after writing this answer, I just saw the comment by whuber about "Grouping hospitals is tantamount to an assumption of homogeneity that ought to be checked." which is what I'm suggesting you address here.]

If the latter is the case, then you might consider taking an approach that is based around having clustered data, wherein the results from individual clusters (hospitals) are then combined to come up with an "overall" pattern of results.

Before I waffle on about clustering, given the huge number of hospitals you have these following methods may not majorly influence the results obtained compared to ignoring hospital (pooling results across all hospitals by age group, and then analysing as though all from a single sample.) This approach would in part depend on the relative sizes of the different hospitals (e.g. if there is a hospital serving a massive population that has quite different practices to other hospitals, then this hospital will dominate the results obtained).

Conceptually you could think of this as being a meta-analysis of the age -> treatment relationship, where each hospital is treated as a unique "study".

In practice, one could:

1. run some kind of multilevel model for this,

2. use tools designed for analysing clustered survey data (and specifying hospital as a cluster unit, with the regression model then looking at the age -> treatment decision relationship,)

3. calculate summary statistics within each cluster (hospital) -- for example, odds ratios between age groups -- and then use the mean and standard deviation of these odds ratios across all 800 hospitals to construct a confidence interval for the odds ratios. One could similarly calculate proportions per age group per hospital, then use similar methods to get some descriptive statistics around the proportions.

The following link (from Martin Bland) talks about randomised controlled trials, but most of the principles still apply to observational data:

http://www-users.york.ac.uk/~mb55/talks/clusml.htm

Two outstanding points: You'll probably still run into problems with zero-counts or sparse cells in each hospital/age-group combination if your actual data look like the example (you will have odds ratios comparing groups on a per-hospital basis that are zero or infinity). There is also a second issue around treating the given/contraindicated/refused as independent outcomes -- I might have to pass on addressing this part for the moment.

-
 Thank you. This is helpful. I am interested primarily in the age group element, but please note that for each hospital-age group combination, I only have aggregated data. – Robert Long Jan 14 at 13:23 Data aggregated as per your example (i.e. summarised for each age group at each hospital) is all you need for this analysis. If you're doing a summary-statistics analysis, then you can do it directly off that original data (e.g. proportion given treatment in Age Group A in Hospital 1 is 3/5 = 0.6, for same Age Group in Hospital 2 is 7/10 = 0.7.) – James Stanley Jan 14 at 19:55 Note that depending on your stats package, if you want to take a regression-like model approach (survey approach or multi-level analysis), you may need to make line listed data (e.g. create 10 lines for Hospital 1 with a binary variable called TREATMENT, 6 of which have a 1 and 4 given a zero, and an indicator variable for HOSPITAL.) This is relatively simple to do, programmatically speaking (e.g. in R, SAS, Stata.) – James Stanley Jan 14 at 19:58

One simple alternative is to make a contingency table (the table whould be different for each of your hypothesis) and then do a chi-square test. Or you could do the binomial regression. In this simple example this ought to give similar results.

-
Maybe sparse cells would make the chi square test invalid. Do you have enough information about the proposed table to back up your assertion that the two methods would give similar results? – Michael Chernick Oct 9 '12 at 17:26
But if there is too small counts, one could use the simulation implementation of Fisher exact test as implemented in R. Too small counts would give problems for the asymptotics behind the logistic regression p-values also! – kjetil b halvorsen Oct 9 '12 at 19:37
Yes of course Fisher's exact test is an option. I use SAS which gives chi-square and Fisher's exact in PROC FREQ. The point is you didn't mention it. – Michael Chernick Oct 9 '12 at 20:19
...and the "etc." in the question suggests the table may be too large to compute the p-value very well, either. (cc @Michael Chernick.) – whuber Oct 9 '12 at 21:55
OK: Your question does not indicate how many age groups you have, nor is it clear that you are willing to group hospitals--please consider editing it to reflect these facts. Grouping hospitals is tantamount to an assumption of homogeneity that ought to be checked. Fisher's test on an 800 by 5 by 2 table would take a bit of computation :-). – whuber Oct 10 '12 at 14:46