I have data from a psychology experiment in which participants were assigned to one of 3 training conditions and then gave responses for 16 trials for each combination of two within-subjects factors, section and format. Each response is classified as correct or incorrect. I originally calculated a % correct for each participant for each combination of section and format, then used a mixed ANOVA to analyze these percentage scores with condition as a between-subjects factor and section & format as within-subjects factors. However, a reviewer of my manuscript commented that ANOVA is inappropriate for this accuracy data because it's not a genuine continuous variable and has various properties making it inappropriate to use ANOVA. I think logistic regression WOULD be applicable to this data because it is, after all, binary choice data originally. Assuming that's right (please tell me if not), how can I do this in R?
Specifically, I need to do logistic regression with correctness/incorrectness of responses as my DV, condition as a b-s factor, and section & format as w-s factors. (I do not have any continuous predictors.) Here is what my data looks like:
nsubj = 150
nsection = 3
nformat = 3
ntrials = 16
subjid = rep( 1:nsubj, each=nformat*nsection )
condition = c( replicate( nsubj, rep( sample( c( 'condition 1', 'condition 2', 'condition 3' ), 1 ), nformat*nsection ) ) )
section = rep( rep( c( 'a', 'b', 'c' ), each=nformat ), nsubj )
format = rep( rep( c( 'x', 'y', 'z' ), nsection ), nsubj )
nCorrect = sample( 1:ntrials, nsubj*nsection*nformat, replace=TRUE )
nIncorrect = ntrials - nCorrect
D = data.frame( subjid=factor(subjid), condition=factor(condition), section=factor(section), format=factor(format), nCorrect=nCorrect, nIncorrect=nIncorrect )
D$accuracy = D$nCorrect / (D$nCorrect+D$nIncorrect)
I tried this, but I know it is wrong because it does not account for the repeated measures, and may inflate significance in other ways (I keep getting significant results from randomly generated data, which seems wrong):
fit <- glm( cbind( D$nCorrect, D$nIncorrect ) ~ section*format*condition, family=binomial("logit"), data=D )
anova( fit, test="Chisq" )
