Summary: I have a population of people split randomly into seven distinct cohorts, amongst whom events occur each month with varying frequency. We have made the same environmental change across all seven cohorts, and I'd now like to see whether the frequencies since that change show a statistically-significant decrease across the population as a whole or not.
The detail: An analogy would be a year of students split randomly into seven differently-sized classes, and the variable I'm looking at is number of detentions received. I'm interested in the picture across the entire school before/after the environmental alteration, rather than changes within individual classes.
However, I'm struggling to work out how to test for that because of the way the data are set out. Points of note are:
- The seven cohorts are different sizes.
- The people are all part of the same group as a whole, but the data are arbitrarily split into these cohorts [continuing the example above, they're all students, but randomly split into different classes].
- The frequency of event differs between cohorts, i.e. one cohort may see on average five events per month, while another might see twenty-five. This does not bear any relation to how the population has been split into these cohorts, but is a (loosely) function of the size of cohort, as you'd expect.
- The number of monthly samples for each population before and after the environmental change varies between populations - some have ten months (ten measurements) before and ten after, while some have as few as four after.
Is it possible to do a single test which looks at the impact of the environmental change across all populations? Or would I need to do a test against each population and draw conclusions from those collectively?
If I'm honest, I'm struggling to even work out if the data are distributed normally - I'm not sure whether I should be looking at the frequency of events within a single cohort over time, or the total frequency across all cohorts over time. [I guess they would be normally-distributed across all cohorts, but that's as likely to be because some classes are very large, and some are very small, rather than anything to do with the environmental change - does that matter? Can you even confirm a normal distribution when n=7?]
I'm trying to revive my PhD-level stats, but failing miserably! If anyone could point me in the right direction then I can happily plough through the calculations, I'm just struggling to determine which test is appropriate given the complexity of the data. Many thanks.