I am using $\chi^2$ to analyze a set of data gathered over 4 time periods. The two variables are independent--each 'event' is categorized as 'good' or 'bad'. We want to prove that the intervention caused an increase in the good events and decrease in the 'bad' events, or at least that the increase in good events outpaces the bad events.
Here is a sample of what the data might look like:
good bad m1 1035 2278 m2 1152 2643 m3 1189 2917 m4 1125 2974
A $\chi^2$ test returns 15 which would have me believe the result is significant, but the change doesn't look very significant at all. In fact, I don't think there is a case to be made that there wasn't any improvement at all. I think that this may be due to the fact that my $N$ varies in each measurement (though I thought $\chi^2$ controlled for that).
It also seems strange to me that if I simply move the decimal over two places (thereby changing my N but not the proportion of good results to bad) then I get $\chi^2=.15$ which is closer to what I would expect.
I know I need to change my approach drastically. How can I make this comparison? Should I be using two independent student's t-tests? Or maybe just use the correlation coefficient for each column and the coefficient of determination?
Thanks in advance for your help and please let me know if anything is unclear.
Here is a thought: should I set up benchmarks for expected improvement, something like 3-5% increase (proportional to $N$) in each measure? This way my expected values in my $\chi^2$ might more accurately reflect what I am trying to show.
EDIT for clarification:
the % of good events compared to bad over the measures:
31%, 30%, 28%, 27%
Clearly there has not been improvement with regard to the proportion of good to bad so what test should I be using to demonstrate this?