My research question is from education: To test whether groups (university, department, gender) differ based on their scores in a test, and whether there are any interaction effects, I'll do 3-way ANOVA? Data is unbalances (unequal cells).
My professor says I have to do SOS Type 1 (report parameter estimates) because of the unbalanced design. His rationale is to compare the effect of each main effect adjusted for the effects of the other? He also said he'd prefer gender to be the first, he'd want parameter estimates, and earlier he had said he prefered Cohen's d type effect sizes.
1) I put gender,university,department, than all 2way interactions, than 3-way interaction in the model. I found gender was not SS, and no SS interaction of it, what-so-ever. Fgender (1,b) = 1.83, p = .18, η2 = .006. I reported R-squared. Should I have proceeded with all other combinations? There are a lot, right?
I dropped gender and redid the analysis with (2x2) [university, department, uni*dep].
2a) I reported Funiversity (1,b) = 46.45, p < .01, η2 = .16. I looked at the estimated means, reported them, and calculated Cohen's d -0.8 ( the difference between group means divided by the square root of the mean square error). I also reported Unstandardized Buniversity = 0.48, t = 5.67, p < .01. and concluded university A were more likely to get a higher score than university B. I used an online effect size conversion calculator from eta-squared to cohen's d. It also η2 = .16 = d = 0.8.
2b) I redid the analysis with [department, university, uni*dep). I reported the exact stuff above for department. But the η2 = .13 and when I calculated Cohen's d it turned out to be 0.38. I used an online effect size conversion calculator from eta-squared to cohen's d, but it reported 0.70. What is wrong?
2c) I plotted marginal means for uni*dep. I put it there to show there is interaction.
2d) I came to the interaction. I knew it would not change in Type I SOS, so I reported Funiversity by department (1, b) = 11.80, p < .01, η2 = .04.
So I said first two are practically the last is SS.
A) My question is to be sure to go to my prof unembarrassed: Is this okay? What else can I do for the interaction? Can I not say uniBdepB < all others, which is obvious in the graph?
B) If parameter estimates tell me the slope, how come they are not equal to estimated means?
C) How can I interpret the parameter estimates for the interaction effect? Which has a value for uniAdepA but than it says the rest are redundant?