Comparing repeated-measures means with unequal number of datapoints per condition

Question

I have data from a study in which subjects were listening to a musical piece and asked to press a key at certain moments. The time axis has been segmented into adjacent epochs of time (of varying durations) that have, for music analysis purposes, been labelled as type A, B or C. Adjacent epochs are not necessarily of the same type, and there are more type-B and type-C epochs than there are of type-A.

For each subject, I computed a keypress count within each epoch in the piece, and thus obtained cross-subject means for each epoch, several ones for each epoch type (A,B,C).

I would like to do a statistical test between these group means to check for a significant difference between A-B-C, but the problem I see is there being an unequal number of elements (epochs) in each of the 3 groups. I think a repeated-measures ANOVA is therefore probably unsuitable.

I also think based on this past CV post that the unequal-number-of-elements-in-each-group problem can be overcome by computing pair-wise differences when such pairs can be defined, and then running paired-samples t-test on the difference (A vs B, B vs C, A vs C). However, I am not sure this data is truly "paired" since the data points refer to epochs from various parts of the piece; or even really "repeated measures" in the traditional sense!

Does it even make sense to run a statistical test in this case, and if so, which test is appropriate?

Answer 1

Based on your clarifications in the comments, I think you should proceed with a Poisson mixed-effects model, a special case of a generalized linear mixed-effects model (GLMM).

If clicks is the number of clicks a subject, subject, performed on a given epoch, epoch, from the set {A, B, C}, then you could start with a model like this one (I am using the R package lme4):

library(lme4)
glmm = glmer(clicks ~ 1 + epoch + (1|subject), family = 'Poisson')

Since your response variable is counts of a process, number of clicks, then the Poisson distribution can be used as a model. Models based on distributions such as the Normal or $t$, are unlikely to provide a good fit in this case.

This approach can deal with unequal samples between epochs and takes into account the variability across subjects through the random effects term (1|subject). This is the equivalent of performing a paired $t$-test or a repeated-measures ANOVA. The reason you need to include such a term is that variability between subjects is expected to exist and you expect responses of a certain subject to be correlated. If you perform the analysis on subject averages or ignore the subject identification by removing the random effects term, you are throwing away existing information in your data that may or may not change the results and their interpretation.

Models such as this one are called mixed-effects, since they combine both fixed effects, (epoch in this case), for which we want to get an estimate of their effect on the response variable, and random effects, for which we assume that they have no overall influence on the response but contribute towards its variance. You will find plenty of posts in this site for more information on mixed-effects and specifically glmm models.

Without knowing more details about your exact experimental conditions, I would advise to also have a look at whether the sequence of epochs and their duration are important by including them as variables in the model.