I have data from a study in which subjects were listening to a musical piece and asked to press a key at certain moments. The time axis has been segmented into adjacent epochs of time (of varying durations) that have, for music analysis purposes, been labelled as type A, B or C. Adjacent epochs are not necessarily of the same type, and there are more type-B and type-C epochs than there are of type-A.

For each subject, I computed a keypress count within each epoch in the piece, and thus obtained cross-subject means for each epoch, several ones for each epoch type (A,B,C).

I would like to do a statistical test between these group means to check for a significant difference between A-B-C, but the problem I see is there being an unequal number of elements (epochs) in each of the 3 groups. I think a repeated-measures ANOVA is therefore probably unsuitable.

I also think based on this past CV post that the unequal-number-of-elements-in-each-group problem can be overcome by computing pair-wise differences when such pairs can be defined, and then running paired-samples t-test on the difference (A vs B, B vs C, A vs C). However, I am not sure this data is truly "paired" since the data points refer to epochs from various parts of the piece; or even really "repeated measures" in the traditional sense!

Does it even make sense to run a statistical test in this case, and if so, which test is appropriate?

  • $\begingroup$ How many subjects do you have and how many trials did each subject complete? All subjects were listening to the same musical piece and A, B, and C where the same for all subjects, right? Sharing some (sample if you cannot share all of your) data would facilitate getting an answer. $\endgroup$ – vkehayas Jul 1 '18 at 21:52
  • $\begingroup$ There are 32 subjects, who listened to the same piece; epoch types were defined based on absolute positioning within the piece, e.g. 0:12 to 1:25; and are thus the same for all subjects. Note that the observations that I would like to compare category-wise are at epoch level. $\endgroup$ – z8080 Jul 2 '18 at 11:41
  • $\begingroup$ OK, I see. So you mean that each subject heard one song with multiple A, B, and C epochs. Can you share some (toy) data? $\endgroup$ – vkehayas Jul 2 '18 at 14:51
  • $\begingroup$ Thanks for your help, I've created a spreadsheet with the data here. Just to reiterate that my question is really rather conceptual, and the data structure uploaded is very simple indeed: for each epoch type, I have a different number of scores (6 "A" scores, 13 "B" scores, 19 "C" scores), and the question is, does it make sense to claim a significant difference between the three means, when there are unequal numbers of data points contributing to each. Again, each score is computed across subjects, and each subject contributes to each of the 3 epoch types $\endgroup$ – z8080 Jul 3 '18 at 15:25
  • $\begingroup$ The reason I keep asking for the data is that I am afraid the description is not clear enough (at least for me) and the simplest way to understand the problem would be to look at your data. Unfortunately, the data you shared do not have all the necessary information. I was expecting to see a tall table with key variables that are missing, such as time and subject. From what I see in your linked data, am I to guess that each epoch can be A, B and C at the same time? Otherwise what does a row designate? Are these data from a single subject? $\endgroup$ – vkehayas Jul 4 '18 at 8:24

Based on your clarifications in the comments, I think you should proceed with a Poisson mixed-effects model, a special case of a generalized linear mixed-effects model (GLMM).

If clicks is the number of clicks a subject, subject, performed on a given epoch, epoch, from the set {A, B, C}, then you could start with a model like this one (I am using the R package lme4):

glmm = glmer(clicks ~ 1 + epoch + (1|subject), family = 'Poisson')

Since your response variable is counts of a process, number of clicks, then the Poisson distribution can be used as a model. Models based on distributions such as the Normal or $t$, are unlikely to provide a good fit in this case.

This approach can deal with unequal samples between epochs and takes into account the variability across subjects through the random effects term (1|subject). This is the equivalent of performing a paired $t$-test or a repeated-measures ANOVA. The reason you need to include such a term is that variability between subjects is expected to exist and you expect responses of a certain subject to be correlated. If you perform the analysis on subject averages or ignore the subject identification by removing the random effects term, you are throwing away existing information in your data that may or may not change the results and their interpretation.

Models such as this one are called mixed-effects, since they combine both fixed effects, (epoch in this case), for which we want to get an estimate of their effect on the response variable, and random effects, for which we assume that they have no overall influence on the response but contribute towards its variance. You will find plenty of posts in this site for more information on and specifically models.

Without knowing more details about your exact experimental conditions, I would advise to also have a look at whether the sequence of epochs and their duration are important by including them as variables in the model.

| cite | improve this answer | |
  • $\begingroup$ Thanks!! Will study your answer in more detail tomorrow, for now I awarded the bounty as it would otherwise soon expire. Thanks again.. $\endgroup$ – z8080 Jul 4 '18 at 20:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.