chi squared to assess the effectiveness of a treatment?

Question

this is probably a weird situation: I am an assistant in a course teaching research methodology, yet I can't quite come up with an answer that satisfies me...

My students are drafting a mock research proposal - one in particular wants to recruit a number (let's say 60) of subjects with a specific pathological condition, randomly assign them into two groups (treatment, control), measure the trait before and after the treatment and check whether the treatment was effective. So far, so good, right? I have one binomial variable as IV and one binomial as DV. As randomization would lead us to believe that the two groups (treated/controls) are identical AND all subjects have the trait before the treatment, we could focus only on the measurement after the treatment, crosstab the variables and compute a chi squared test for the indipendence (treatment/no treatment; trait/no trait).

However, I have some doubts: 1) I believe that at least one cell would have a value close to zero (no control subject will spontaneously lose the pathological condition), AND I believe that also the vast majority of treated subjects will not improve. Therefore, an exact test should be a better choice

2) Fisher's exact test would take care of the problem above, but I am still not convinced that is the right solution. I believe that computing something like a risk ratio (where the event is 'recovery') would be more informative and appropriate.

But my question is: is solution 2) adequate? For some reason that I can't really pinpoint, I am feeling that I'm missing a piece of the picture. Is that so?

Thanks a lot!

Answer 1

My understanding is that Fisher's Exact Test is inappropriate unless you have fixed margins. Although the treatment margins are fixed, the "recovered" margins are not.

You could easily perform a logistic or loglinear regression to estimate the odds ratio or risk ratio. There is nothing wrong with this approach. Just make sure you interpret the coefficients on the model correctly. You could also run a t-test for proportions. Unlike a chi-square test, these tests provide an interpretable measure of effect size and its confidence interval.