Interpreting interactions between two treatments I have two treatments A & B. Here are my groups, where X represents the appropriate control for that particular treatment:
Group 1: XX
Group 2: AX
Group 3: XB
Group 4: AB
The hypothesis is that the treatment B will have an effect, but that that effect will no longer be apparent when combined with treatment A.
So, if run my experiment and run an ANOVA on the data, and the results of the analysis show that only Group 3 was significantly different than the others, is it correct to say that "treatment B had an effect and that effect was lost when combined with treatment A"? Or, do would I also need to show a significant different between XB and AB?
 A: If I understand you correctly, your design is:
$\begin{array}{rcccl}
~     & B_{X}    & B_{B}    & M     \\\hline
A_{X} & \mu_{11} & \mu_{12} & \mu_{1.} \\
A_{A} & \mu_{21} & \mu_{22} & \mu_{2.} \\\hline
M     & \mu_{.1} & \mu_{.2} & \mu
\end{array}$
The first part of your hypothesis (effect of treatment B within control group of A) then means that $H_{1}^{1}: \mu_{12} - \mu_{11} > 0$.
The second part of your hypothesis (no effect of treatment B within treatment A) would then be $H_{1}^{2}: \mu_{22} - \mu_{21} = 0$.
So your composite hypothesis is $H_{1}: H_{1}^{1} \wedge H_{1}^{2}$. The problem is with the second part because a non-significant post-hoc test for $H_{0}: \mu_{22} - \mu_{11} = 0$ doesn't mean that there is no effect - your test simply might not have enough power to detect the difference.
You could still test the hypothesis $H_{1}': (\mu_{12} - \mu_{11}) > (\mu_{22} - \mu_{21})$, i.e., an interaction contrast. However, this tests the weaker hypothesis that B has a bigger effect within A's control group than within treatment A.
I'm not sure what you mean by "the results of the analysis show that only Group 3 was significantly different than the others". I don't understand how exactly you would test that. You could test $\mu_{12} \neq \frac{1}{3} (\mu_{11} + \mu_{21} + \mu_{22})$, but that is a weaker hypothesis (Group 3 is different from the average of the remaining groups).
A: If in post hoc testing Group 3's mean was significantly different from all the others' then you've already shown that XB is different from AB. Am I missing something?  Your statement about B's effect (and its being lost when combined with A's) would be correct.
