I'm doing a logistic regression in R (using glmer and family='binomial').
I'm measuring how performance in a task differs between 2 factors (2 levels each). The IVs are categorical. (It might be helpful to know that the design is split-plot, with one IV repeated within subjects and subjects nested within the other IV).
I previously did the analysis as a regular GLM by converting the DV to percent correct (i.e. number of items correct / number of items total) for each condition. However, this method does not allow for item to be used as a random effect. It also assumes a continuous distribution for the DV and a distribution that goes above 1.0 and below 0.0, both of which are violated.
Therefore, I chose to use logistic regression and code each item (per participant) as a success or failure (DV is 1 or 0). (Subject and Item are random factors in the analysis).
Interestingly, the logistic regression completely eliminates the interaction effect of the 2 IVs, which was significant in the linear regression. (This is even the case if I make the analyses more comparable by not adding item as a random effect.)
One reason why I think this is happening: One of the 4 categories has a very high proportion of participants with perfect accuracy (i.e. 10 of 10 items correct). It seems that this is disproportionately increasing the Beta estimate for that category.
Looking at a graph of the logit function, the mapping between probability and log odds ratio blows up around p=1.0 or 0.0, with very small changes in p producing very large changes in the log odds ratio. (The R analysis uses a binomial distribution, but I would think this still to be related.)
In cases like these, with a categorical IV and perfect accuracy for one of the categories, is logistic regression still valid? My intuition is that this might make the estimates less robust. If so, what are alternatives to logistic regression that I can perform which will still counter the problems associated with regular GLM?