# Proportion test: one sample or two sample?

We introduced a new way of teaching children this summer. We then split the children (As randomly as possible) into 2 groups. Group 1 children were taught in the new style, Group 2 children were taught in the usual style. (Business as usual). At the end of the summer children in both groups took an exam and we measured what % of children passed that exam.

We want to see if Group 1's children (new instruction method) performed better than Group 2's children (Business as usual).

My question is: should I be conducting a 1 sample test here or a 2 sample test?

In a 2 sample test, I would simply consider the two groups as having received two different treatments (even though group 2 was "business as usual")

In a 1 sample test, I would compare Group 1's data against the pass percentage of Group 2. (think of group 2's pass percentage as fixed)

What would you suggest?

Suppose 78 students out of 110 in the control group passed the test and 92 out of 205 in the treatment group passed.

A one-sided two-sample prop.test is appropriate. In R, two different patterns of syntax are possible to present the data. Both are illustrated below.

Both methods result in the same P-value 0.002235, which is smaller than 5%. So you would reject at the 5% level the null hypothesis $$H_0$$ that control and treatments groups have essentially the same pass rate, against the alternative $$H_a$$ that the pass rate for the control group (mentioned first) has a pass rate that is 'less' than the pass rate for the treatment group.

x1 = 78;  n1 = 110;   x2 = 92;  n2 = 105
prop.test(c(78,92), c(110,105), alt="less")

2-sample test for equality of proportions with continuity correction

data:  c(78, 92) out of c(110, 105)
X-squared = 8.0822, df = 1, p-value = 0.002235
alternative hypothesis: less
95 percent confidence interval:
-1.00000000 -0.06908546
sample estimates:
prop 1    prop 2
0.7090909 0.8761905


Alternatively,

x1 = 78;  n1 = 110;   x2 = 92;  n2 = 105
g1 = c(x1, n1-x1);  g2 = c(x2, n2-x2)
TBL = rbind(g1, g2);  TBL   # 'bind' two row vectors to make table
[,1] [,2]
g1   78   32
g2   92   13
prop.test(TBL, alt="less")

2-sample test for equality of proportions with continuity correction

data:  TBL
X-squared = 8.0822, df = 1, p-value = 0.002235
alternative hypothesis: less
95 percent confidence interval:
-1.00000000 -0.06908546
sample estimates:
prop 1    prop 2
0.7090909 0.8761905


Note: If you don't want to use the Yates continuity correction (skipping it is my preference), then use argument cor=F in function prop.test. The continuity correction is biased toward making the chi-squared statistic smaller and the P-value larger. (There is disagreement whether use of the continuity correction is warranted.)

prop.test(TBL, alt="less", cor=F)\$p.val
[1] 0.001303608


TWO-SAMPLE

TWO-SAMPLE

TWO-SAMPLE

Your group 2 is called a control group. Without a control, you do not know the reason for any changes in performance. Sure, it could be that the new method is more effective. It also could be that you gave a particularly easy exam. Or maybe you keep using the same exam but the questions are getting leaked after a few years.

The control group gives you a baseline measurement against which you can make comparisons.

• Great, thanks. Yep - I know, test vs control. I'd still compare the two. If using the 1 sample, I guess the data being lost is the variance in the control sample. Right? Oct 8, 2019 at 23:26