Using R's prop.test function to compare subsets of a population I'm new to CV, so please excuse any faux pas. I've collected sample data from a population and would like to discern whether or not the experimental group outperforms the control group. Let's say I have 10,000 observations within the experimental group and saw 4500 successes. Let's also say that I observed a 47% success rate in my control group (so if my control group is 5% of the total sample size, that would be 235 successes out of 500 trials).
Now, am I right in my usage and interpretation of the following?:
prop.test(4500,10000,.47,alt="less")

data:  4500 out of 10000, null probability 0.47
X-squared = 15.9776, df = 1, p-value = 3.205e-05
alternative hypothesis: true p is less than 0.47
95 percent confidence interval:
 0.0000000 0.4582455
sample estimates:
   p 
0.45

Since the p-value is far less than .05, I can safely reject the null hypothesis that that the experimental group will perform as well or better than the control group. Also, I know that 95% of the time, the probability of success for the experimental group will fall between 0% and ~45.8%.
Also, is the prop.test function a good indicator of sufficient sample size? If not, what is?
Thanks!
 A: Your interpretation is incorrect.
"Since the p-value is far less than .05, I can safely reject the null
hypothesis that that the experimental group will perform as well or
better than the control group. "
No. What you did was this: you "tested" if the p from the experimental group is greater or equal 0.47. But you don't know if the p of the control group is actually 0.47. So you could say that the evidence favors the hypothesis that the proportion in the experimental group is less than 0.47, but you could not say that the evidence shows that it is less than the proportion of the control group, which is uncertain.
Actually, just for illustration, assuming the values stated on the question, you wouldn't reject the hypothesis that the proportions are different with prop.test:
success <- c(4500, 235) ## successes in experimental and control
n <- c(10000, 500) ## total observations in experimental and control
prop.test(success, n)

2-sample test for equality of proportions with continuity correction

data:  success out of n
X-squared = 0.6907, df = 1, p-value = 0.4059
alternative hypothesis: two.sided
95 percent confidence interval:
 -0.06587065  0.02587065
sample estimates:
prop 1 prop 2 
  0.45   0.47 

Notice that once you account for the uncertainty in the other proportion the p-value raises to 0.4.
As for the second sentence:
"Also, I know that 95% of the time, the probability of success for the experimental group will fall between 0% and ~45.8%."
This is a common misconception. What you can say is that the procedure you used constructs confidence intervals that would cover the true value of p 95% of the time. But this is not the same as claiming that 95% of the time you will estimate p on the range of the confidence interval, neither that the true p is inside the confidence interval with 95% probability. This other question may help on this topic.
