Should I include this random effect in a mixed model? I'm rephrasing a question I asked before - any input would be hugely appreciated, as I haven't found similar examples online.
Consider a design where random effect 'a' is fully crossed with random effect 'b' and random effect 'c' is fully crossed with random effect 'd'. Random effect 'c' is nested within a x b combinations. 
 
'a' are language editors, 'b' are sentences to be edited, 'c' are the edited versions of these sentences (i.e. edited by 'a'), and 'd' are judges who provided quality scores on 'c', the edited versions. Each instance of 'c' received a different ID in the data, though they may turn out to be identical (i.e. in case editors edit the same instance of 'b' in the same way). The DV is quality scores related to 'c' (provided by 'd') and predictors relate to 'a' and 'b'.
My questions are:
- Do I need 'c' as a random effect in the model, when each instance of 'c' is already represented by a x b combinations?
- Is it OK to give different IDs to each 'c' even when there's the possibility of these being identical (i.e. if the same 'b' happens to be edited in the same way by different editors)?
 A: Adding a label to each distinguishable edited sentence in set 'c' might provide a way to evaluate intra-judge variability in scoring, based on each judge's different ratings of the quality of the same sentence. Note that this labeling of distinguishable sentences doesn't add anything to evaluating the upstream issues of the input sentence-editor-output sentence process, your primary interest.
You could incorporate this intra-judge variability in your model, but you don't have much evidence for such intra-judge variability, and it doesn't seem to be of much concern to you. In that case there really is no need to add this extra complexity to what seems otherwise to be a nicely balanced crossed design, and would probably cost you unnecessary degrees of freedom.
If you want the best estimate of inter-judge variability, you will have to code things in a way that you compare responses of all judges to each combination of input sentence and editor. If the inter-judge variability is not of concern, and you simply want to average scores among the judges and perhaps correct for differences in mean scores among judges, then you don't have to code the individual output sentences. So the answer to your question depends on how much you want to know about inter-judge variability.
