I currently have a 2x2 table of proportions for treatment effects that looks like:
\begin{array}{lll} & \text{Treatment } A & \text{Without Treatment } A \\ \text{Treatment } B & 15/250 & 8/250 \\ \text{Without Treatment } B & 5/250 & \end{array}
I am trying to test to see if there is a synergistic effect of Treatment A and B. In this case, would a chi-square test work best or would a $2^2$ factorial design work?
With a normally distributed response a simple way to do this is to run it as a 'flat' / one-way ANOVA. Then you would run a-priori orthogonal contrasts: {A~B vs. ~AB}, and then {A~B&~AB vs. AB}. I gather these may be binomial, so you could do the analogous thing with logistic regression, or even a set of chi-squared analyses.
Here is an example walkthrough using your data, with R:
d = data.frame(s=c(5 ,8, 15), f=250-c(5 ,8, 15), treat=c("AnB", "nAB", "AB"))
summary(glm(cbind(s,f)~treat, family=binomial, d))
# ...
# Coefficients:
# Estimate Std. Error z value Pr(>|z|)
# (Intercept) -2.7515 0.2663 -10.332 <2e-16 ***
# treatAnB -1.1403 0.5244 -2.174 0.0297 *
# treatnAB -0.6580 0.4473 -1.471 0.1413
# ...
# Null deviance: 5.7452e+00 on 2 degrees of freedom
# Residual deviance: -8.5487e-14 on 0 degrees of freedom
1-pchisq(5.7452e+00--8.5487e-14, df=2-0) # [1] 0.0565517
Note that the p-values listed with the coefficients are Wald tests, and that with multiple categories (as here with three), they do not quite tell you if the conditions differ, but only if the indexed condition differs from the reference condition. To test whether there are differences amongst the three conditions, I have performed likelihood ratio tests using the null and residual deviances and degrees of freedom. By conventional criteria, the treatments do not quite differ significantly.
summary(glm(cbind(s,f)~treat, family=binomial, d[-3,]))
# ...
# Coefficients:
# Estimate Std. Error z value Pr(>|z|)
# (Intercept) -3.8918 0.4518 -8.615 <2e-16 ***
# treatnAB 0.4823 0.5772 0.836 0.403
# ...
# Null deviance: 7.1707e-01 on 1 degrees of freedom
# Residual deviance: -1.2657e-14 on 0 degrees of freedom
1-pchisq(7.1707e-01-1.2657e-14, df=1-0) # [1] 0.3971067
(In this case, you could use the Wald test, but I ran a likelihood ratio test for consistency.) There is not enough evidence to conclude the $A\neg B$ and $\neg AB$ conditions differ.
summary(glm(cbind(s,f)~I(treat=="AB"), family=binomial, d))
# ...
# Coefficients:
# Estimate Std. Error z value Pr(>|z|)
# (Intercept) -3.6233 0.2810 -12.893 <2e-16 ***
# I(treat == "AB")TRUE 0.8718 0.3872 2.252 0.0243 *
# ...
# Null deviance: 5.74520 on 2 degrees of freedom
# Residual deviance: 0.71707 on 1 degrees of freedom
1-pchisq(5.74520-0.71707, df=2-1) # [1] 0.02493881
The data are not consistent with the assumption that $AB$ is the same as $A\neg B\cup\neg AB$.
One can consider these as 3 groups: A, B, Both
These can then be analyzed with Chi-Square test:
A B Both
Event 5 8 15
NoEvent 245 242 235
================= Chi-Square Test =================
chi2: 5.86; dof: 2
p value: 0.053
The resulting P-value is of borderline significance.
Also, one can combine groups A and B and compare them together with group AB using Chi-squared test or Fisher's exact test:
> mm2
Response No_response
Tt_A_or_B 13 487
Tt_AB 15 235
> chisq.test(mm2)
Pearson's Chi-squared test with Yates' continuity correction
data: mm2
X-squared = 4.4566, df = 1, p-value = 0.03477
>
> fisher.test(mm2)
Fisher's Exact Test for Count Data
data: mm2
p-value = 0.02496
alternative hypothesis: true odds ratio is not equal to 1
95 percent confidence interval:
0.1801596 0.9605233
sample estimates:
odds ratio
0.4187352
Hence, combination treatment (A+B) is statistically better than single treatment (A or B).
