Is a repeated measures ANOVA appropriate for multiple within-subject measurements?

Question

I recorded 4 groups of captive birds (each group a difference species) at the same 3 times each day for 15 days. I want to determine if there is a difference in their vocalization properties such as call rate (the number of chirps per second for each call), call length, etc. during these 3 specific times of day. I repeated my recordings over several days because of the varying number of calls - sometimes the birds were silent, other times there were 10 calls in one recording session, etc.

1. My first reaction is to run a RM ANOVA (in SPSS), and if the differences between time of day are significant, do post-hoc tests for further investigation. Does this seem appropriate? I was initially going to do individual tests for each group, but now I am wondering if I can do the RM ANOVA using all of the groups and controlling for species differences.

2. I was thinking of just analyzing the actual calls that were recorded, ignoring the fact that some days there were no calls to analyze, as I am not interested in the differences between days, only between times. The problem with this is that I then have unbalanced data - for example, I may have a total of 22 calls to analyze at the beginning of the day, 17 at the middle, and 35 at the end. Would this be an issue with a RM ANOVA?

I have been confusing and second-guessing myself like crazy - so I appreciate any advice!

Answer 1

The data

You have count data. Each observation is therefore something like '10 calls at time point 1 on day 4'. This has two implications:

• Unless you forgot to record at one of the 45 day/time combinations, your data are balanced. This is true even if there were no calls at some combination. In that case the observation is simply 0.

• Unless the counts are high with very similar rates, the 'sphericity' assumption (i.e. that all 45 observations have the same standard deviation) that @gung mentions is unlikely to hold.

You might therefore be better switching to a generalized linear regression framework that will allow you to model the counts as conditionally Poisson. If you really don't want to there are various transformations of your counts (e.g. square root) that might be useful to keep you in a broadly in ANOVA's linear normal framework. It may or may not be worth getting hung up about this. It's hard to tell without playing with the data. Try things a couple of ways and check your residuals.

Random effects / Repeated measures

If you consider that some days are just noisier than others and that it is scientifically irrelevant which particular days you chose to record on, but scientifically interesting to see whether and how measurements at time points 1 2 and 3 differ from each other, then it seems quite reasonable to me to treat the day of recording as a random effect and the time of recording as a fixed effect.

The interpretation would be that each day has some intercept relating to the average call rate which is a random draw from the population of all the days you could have recorded, and the time you recorded adds or subtracts something from this rate. The observation is a random draw from a distribution with this mean. You're then asking whether the time of recording stands out from the day variation.

Answer 2

It sounds like your recordings are all of the same group of captive birds, and you're not distinguishing among the calls of individual birds in this group. If this is the case, you effectively have data from one "subject," so you'd need another way to control for differences among individual birds in your group. Lacking this, there's no justification for RM ANOVA; you can't separate subjects error from total error when you only have one subject—the group as a whole. I guess you could treat the 15 days as different subjects and try to separate noisy-vs.-quiet-day variance from time-of-day variance, but if consecutive days are more similar than widely separated days (e.g., if the first day is more similar to the second day than the last day), you'd still lose the ability to control for that by using RM ANOVA.

I think the typical, simple analysis for this kind of problem is the chi-squared goodness of fit test. Basically, this tests how well your data fit a given model of expected frequencies. It sounds like you'd want to test the usual, default model, which would represent the expectation that calls occur equally often at all three times of day. If the test produces a significant result, you can reject this model as a null hypothesis, and conclude that calls are likely to occur with different frequencies at one or more of your times of day. Using this test, you can also test different models than simple equality, which you may wish to do if you have particular theories you'd like to falsify. However, Wikipedia's page on this test indicates your results won't be reliable if even one of your three cells' expected frequencies is less than 5. Null hypotheses aside, Cramér's V should work in this case as an effect size estimate for the similarity of the frequencies across your three times of day if you're willing to just pool all your different days' observations into three cells of frequencies for each particular time of day.

If you do want to separate out time-of-day effects from other time effects (e.g., differences between days, relationships between consecutive days), your analyses will have to be more complex than the chi-square goodness of fit test. If you're willing to do more complex analyses, a more ambitious answer than mine might be able to serve that aim. In such a case, I'd recommend editing your question to specify what kinds of effects you want to separate from one another, or commenting on my answer to specify its shortcomings for your purposes.