# Finding statistical significance between two binary tests

I have two sets of data. One with a sample size of 82 with 53 "hits" and 29 "Misses." And a second sample of 105 with 67 "hits" and 38 "misses"

Given that the second set of data is a control, is there a way to show that the results of the first data set are or are not statistically significant?

Thanks in advance, It's been a while since I took a stats course and googling for the answer left me confused.

• Is the null that the population proportion of hits is the same for the two groups? Then you have a two-sample proportions test (sometimes two-sample binomial proportions test); it might also be done as a chi-square. Dec 12 '13 at 5:59
• I believe so, the null would be that proportion of hits is the same. I am hoping that the first sample has more hits and isn't just chance. Dec 12 '13 at 6:05

I would start by stating things more formally, e.g. (with no prior information):

The null hypothesis is that these two samples come from the same population. The alternative is that they come from different populations where one really has more "hits".

A simple and traditional approach would be Fisher's exact test.

Wiki should allow you to calculate it manually for your data.

If using R you could do something like this:

> cm1 <- matrix(c(53, 67, 29, 38),
nrow = 2,
dimnames = list(c("first", "control"), c("hits", "misses")))
hits misses Sum
first     53     29  82
control   67     38 105
Sum      120     67 187
> fisher.test(cm1)


Gives p=1 meaning probability of your data given the null is very high (suggests that first group v. unlikely to be different from control).

You can also do

fisher.test(cm1, alternative="greater")


meaning

The null hypothesis is that these two samples come from the same population. The alternative is that the first comes from a different population which has more "hits" than the second.

... but it's still p=0.52.

Now there are other methods but most will accept that it's unlikely that there's any great difference between 53/82 = 65% and 67/105 = 64%

• The two sample proportions test was quite similar, though I did it two tailed. Dec 12 '13 at 10:28
• Really nice answer from dardisco, I don't have enough reputation so I will leave the link to python reference here docs.scipy.org/doc/scipy/reference/generated/… Mar 31 at 11:14