We have carried out an experiment that measures how many references it takes within a group to find the most expert person in that group on a specific topic (DV). We have 7 topics (4 of which are considered "visible" and 3 of which are considered "invisible"). The visible/invisible distinction is our treatment. We have 42 groups (school classes). We conducted the experiment with all 7 topics in each group so that groups and topics are crossed. We believe that the number of references required to find the most expert person in a class depends on the visibility/invisibility of the search topic. I am struggling to come up with the proper design. I believe that search topics are nested in the treatment (visible/invisible). I would also like to consider the fact that I have 7 measures for each class. I am not interested in assessing a direct effect of class but just want to control for the fact that the 7 measures within one class may not be independent from each other. Can this still be done with an ANOVA (and if so what type?) or should I rather use a GLM?
First, thanks for providing context, that always helps
Second, your dependent variable is a count (number of references); you should account for that in your choice of analysis (if the counts were all quite large, this might be ignorable, but it looks like your counts are low).
Third, since you did the experiment multiple times in each class, therefore your data are not independent and you should deal with that in your analysis as well.
One choice would be a nonlinear mixed model, these are quite complex models; if you have little experience with statistics they may be kind of daunting. That said, if you are using
R you can look at the
nlme packages. If you are using
GLIMMIX is the PROC to at least start with.