Suppose that I have two approaches to a particular problem. Approach A is observed to succeed 685 times out of 1347 attempts. Approach B is observed to succeed 2100 times out of 3748 attempts. I want to see if Approach B is preferable to Approach A.

In R I run:

prop.test(c(2100,685), c(3748,1347), alternative="greater", correct=FALSE)

and I get:

data:  c(2100, 685) out of c(3748, 1347)
X-squared = 10.7124, df = 1, p-value = 0.0005321
alternative hypothesis: greater
95 percent confidence interval:
0.02568765 1.00000000
sample estimates:
  prop 1    prop 2
0.5602988 0.5085375

I have the following questions.

  1. Let $p_A$ (resp. $p_B$) be the proportion of successes of Approach A (resp. B). Implicitly, my null hypothesis is that $p_A=p_B$, right? And then, since the $p$-value is less than the significance level, I reject the null hypothesis. To me this seems that I am led to conclude that $p_A \neq p_B$. Of course, it seems likely by just looking at the data that $p_B>p_A$.

  2. Is the syntax alternative="greater" included to rule out $p_A>p_B$?

  3. Similarly, does the line about the confidence interval mean that the "true" value of $p_B-p_A$ lies in the interval $(.02568765,1)$?


This issue isn't really specific to prop.test(), the issue is more generally about the nature of one-sided hypothesis tests. In light of that, you should probably read @whuber's answer here: Justification of one-tailed hypothesis testing.

  1. The gist of the issue is that your null encompasses all possible differences less than or equal to $H_0\!: p_A = p_B$. In order to rule them all out, you need to rule out the most reasonable possibility, namely: $H_0\!: p_A = p_B$. If $p_B$ is sufficiently greater than $p_A$ that $H_0\!: p_A = p_B$ is no longer believable, then $H_0\!: p_A > p_B$ is not a reasonable possibility either.

  2. Yes, the syntax alternative="greater" rules out $H_0\!: p_A > p_B$ as well. Relatedly, it also causes R to only use one tail of the null distribution to assess the p-value.

  3. Yes, the confidence interval refers to the "true" value of $p_B-p_A$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.