Let's say I have an experiment where I randomly pair people up with either a male or a female player, and they play a game together. I then measure some dependent variable -- call it
outcome . From the perspective of participants, there are four possible subgroups -- Male-Male (MM), Male-Female (MF), Female-Male (FM), and Female-Female (FF). I have created a
group variable that is simply a factor variable representing each of those levels.
In my current set-up, if I want to compare differences in means between these groups, I can do something straight-forward like:
felm(outcome ~ group | team_id | 0 | 0)
Where I cluster the standard errors at the team level. However, now I have two control conditions where I pair people up in the same way (into the four sub-groupings), but have them do different activity. I want to demonstrate that, for the same sub-group, the treatment condition has a greater mean than any of the control conditions.
In other words, I want to compare MM in the treatment group to MM in either of the two controls, and confirm that the DV of MM in the outcome group is larger than both of the control conditions.
What is the right way to do this?
My current approach is to create a
condition variable that is equal to three levels, e.g. ("Main Treatment", "Control 1", "Control 2"). So treatment is a factor with four levels ("MM", "FM", "MF", and "FF"):
lm(outcome ~ treatment*condition)
But I'm not sure this is actually capturing what I intend for it to capture
With any of the above approaches, how can I continue to do the relevant comparisons while clustering standard errors at the team level (e.g. via
lfelibrary or lm_robust from
With the ANOVA approach described in the answer, the resulting output will look something like this:
Response: outgroup_feelings_diff Df Sum Sq Mean Sq F value Pr(>F) treatment 3 1276 425.23 3.8863 0.009073 ** condition 2 331 165.67 1.5141 0.220867 treatment:condition 6 1132 188.62 1.7238 0.113025 Residuals 592 64777 109.42
However, this doesn't tell me if a specific combination of the treatment and condition is better than some other combination -- it is presumably averaging across all combinations. Is there a way, in the ANOVA framework to ask some specific question like whether MF in the "Main Treatment" is greater than MF in "Control 1" and "Control 2." (Should I just stick with the Tukey approach if I want to do this?)
- What is the substantive difference between the interaction approach described in #2 and creating an entirely new variable that explicitly interacts treatment and condition. For example:
treatment_condition <- paste0(treatment, condition) lm(outcome ~ treatment_condition)
And then cycling through the relevant reference groups until I get the comparison I want.