I am working with a rather large data set (over 500 subjects) that has been scored/coded for multiple different variables. This particular data set is a longitudinal analysis (with 4 time points) of performance in a reading task with both a clinical population and a non-clinical population. The reading task also has over a dozen different stimuli used. Performance on the task was scored as a continuous integer (raw number of tokens included) for a dozen different types of non mutually exclusive responses. That is, a single subject received a score for each of the 12 Response Types that could range from 0 (no tokens of that type given) to any whole number more than 0 (e.g. 3 tokens for Type A, 5 tokens for Type B, 1 for Type C, and so on).
What I am interested in is how the amount of tokens in each Response Type (A through L) vary across the two populations (Diagnosis) over each of the 4 time points. The trick is that the clinical population gives a shorter response and thus, raw scores are not very helpful in assessing differences in TYPE independent of response length. There are also likely stimuli effects and some subject variability, as well as a co-variate of IQ. Thus, I am fairly certain a mixed model with a DV of Number of Tokens, fixed effects of ResponseType, TimePoint, and Diagnosis, and random effects of word and subject is the best approach. I suspect that I ought to also include a third random effect of ResponseType by Diagnosis (as I want to partial out the variance caused by an overall longer response in the non-clinical sample). However, I'm not sure if that final random effect (ResponseType|Diagnosis) is correct, or if the mixed model is the most optimal approach for this complicated data set (I've previously used just simple models and ANOVAs). I'm worried about oversaturating the model and uncertain on running follow-up tests.
Footnote: Model notation thus far (written for use of lmer in R): NumTokens~ResponseTypeDiagnosisTimePoint+IQ+(1|Stimuli)+(1|Subject)+(1+ResponseLength|Diagnosis). Also, I have already compared this model with simpler models and the resulting comparison indicates this full model is the best fit - removing any variables (fixed or random) decreases the fit significantly.