Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I have three groups of data, each with a binomial distribution (i.e. each group has elements that are either success or failure). I do not have a predicted probability of success, but instead can only rely on the success rate of each as an approximation for the true success rate. I have only found this question, which is close but does not seem to exactly deal with the this scenario.

To simplify down the test, let's just say that I have 2 groups (3 can be extended from this base case).

  • Group 1 trials: $n_1$ = 2455
  • Group 2 trials: $n_2$ = 2730

  • Group 1 success: $k_1$ = 1556
  • Group 2 success: $k_2$ = 1671

I don't have an expected success probability, only what I know from the samples. So my implied success rate for the two groups is:

  • Group 1 success rate: $p_1$ = 1556/2455 = 63.4%
  • Group 2 success rate: $p_2$ = 1671/2730 = 61.2%

The success rate of each of the sample is fairly close. However my sample sizes are also quite large. If I check the CDF of the binomial distribution to see how different it is from the first (where I'm assuming the first is the null test) I get a very small probability that the second could be achieved.

In Excel:

1-BINOM.DIST(1556,2455,61.2%,TRUE) = 0.012

However, this does not take into account any variance of the first result, it just assumes the first result is the test probability.

Is there a better way to test if these two samples of data are actually statistically different from one another?

share|improve this question
Another question I came across that didn't really help much: stats.stackexchange.com/questions/82059/… –  Scott Aug 28 '14 at 17:15
Does this question help? stats.stackexchange.com/questions/25299/… –  Eric Aug 28 '14 at 17:32
In R, you could use prop.test: prop.test(c(1556, 1671), c(2455, 2730)). –  COOLSerdash Aug 28 '14 at 17:48
@COOLSerdash you might want to give the answer in excel since that is what they seem to be using. –  Dan Aug 28 '14 at 17:50
@Scott here is a link on how to perform a Chi Square test in excel: office.microsoft.com/en-us/excel-help/… –  Dan Aug 28 '14 at 17:51

4 Answers 4

up vote 4 down vote accepted

The soultion is a simple google away: http://en.wikipedia.org/wiki/Statistical_hypothesis_testing

So you would like to test the following null hypothesis against the given alternative

$H_0:p_1=p_2$ versus $H_A:p_1\neq p_2$

So you just need to calculate the test statistic which is

$$z=\frac{\hat p_1-\hat p_2}{\sqrt{\hat p(1-\hat p)\left(\frac{1}{n_1}+\frac{1}{n_2}\right)}}$$

where $\hat p=\frac{n_1\hat p_1+n_2\hat p_2}{n_1+n_2}$.

So now, in your problem, $\hat p_1=.634$, $\hat p_2=.612$, $n_1=2455$ and $n_2=2730.$

Once you calculate the test statistic, you just need to calculate the corresponding critical region value to compare your test statistic too. For example, if you are testing this hypothesis at the 95% confidence level then you need to compare your test statistic against the critical region value of $z_{\alpha/2}=1.96$ (for this two tailed test).

Now, if $z>z_{\alpha/2}$ then you may reject the null hypothesis, otherwise you must fail to reject the null hypothesis.

Well this solution works for the case when you are comparing two groups, but it does not generalize to the case where you want to compare 3 groups.

You could however use a Chi Squared test to test if all three groups have equal proportions as suggested by @Eric in his comment above: " Does this question help? stats.stackexchange.com/questions/25299/ … – Eric"

share|improve this answer
Thanks @Dan. As many times with Google, knowing the right term to search for is the first hurdle. I did take a look at the chi-squared test. The problem there, as with where I was first getting stuck, is that my expected calculation is based on the sample. I can't therefore provide an expected value, because my samples are used to determine that expected value. –  Scott Aug 28 '14 at 18:33
@Scott, if your hypothesized proportions for the three groups are that they are all equal then the expected value should be 1/3 for each group. –  Dan Aug 28 '14 at 18:35

In R the answer is calculated as:

fisher.test(rbind(c(1556,2455-1556), c(1671,2730-1671)), alternative="less")
share|improve this answer
Would you consider writing a little bit more than providing the R function? Naming the function does not help in understanding the problem and not everyone use R, so it would be no help for them. –  Tim Dec 8 '14 at 13:52
This is the most exact statistical answer, and works for small numbers of observations (see the following: itl.nist.gov/div898/handbook/prc/section3/prc33.htm). –  Andrew Mao yesterday

Your test statistic is $Z = \frac{\hat{p_1}-\hat{p_2}}{\sqrt{\hat{p}(1-\hat{p})(1/n_1+1/n_2)}}$, where $\hat{p}=\frac{n_1\hat{p_1}+n_2\hat{p_2}}{n_1+n_2}$.

The critical regions are $Z > \Phi^{-1}(1-\alpha/2)$ and $Z<\Phi^{-1}(\alpha/2)$ for the two-tailed test with the usual adjustments for a one-tailed test.

share|improve this answer
Your formula for $Z$ has a mistake. You forgot the square root. –  Dan Aug 28 '14 at 17:46
Embarrassing - thanks @Dan! Fixed now. –  abaumann Aug 29 '14 at 11:11

In hospitals, assuming that the probability of a patient to survive from typhoid is 4/5 or 0.8 meaning out of five typhoid patients only one will not survive. In probability distribution, p=1/5 while q which is also =1-p =4/5. In the case the probability 1/5 represents a success while 4/5 represents a failure. A success is the number of patients who will die of typhoid while a failure is the number of patient who will survive from typhoid Which are given by 1/5 and 4/5 respectively. Now assuming that in the same hospital on a certain day 1 of the twenty patients died and on that day no patient was admitted of typhoid, from this we will take that out of every 20 patients on will die hence the probability is 1/20.this represents a success while 19/20 represents a failure. From this on another day when 100 patients are admitted on the same hospital with twenty being typhoid patients the probability that a patient will die on that day is 3/3*4/20= 12/60+ 3/60= 1/4.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.