I have two sets of data. One with a sample size of 82 with 53 "hits" and 29 "Misses." And a second sample of 105 with 67 "hits" and 38 "misses"

Given that the second set of data is a control, is there a way to show that the results of the first data set are or are not statistically significant?

Thanks in advance, It's been a while since I took a stats course and googling for the answer left me confused.

  • 2
    $\begingroup$ Is the null that the population proportion of hits is the same for the two groups? Then you have a two-sample proportions test (sometimes two-sample binomial proportions test); it might also be done as a chi-square. $\endgroup$
    – Glen_b
    Dec 12 '13 at 5:59
  • $\begingroup$ I believe so, the null would be that proportion of hits is the same. I am hoping that the first sample has more hits and isn't just chance. $\endgroup$
    – JesterF
    Dec 12 '13 at 6:05

I would start by stating things more formally, e.g. (with no prior information):

The null hypothesis is that these two samples come from the same population. The alternative is that they come from different populations where one really has more "hits".

A simple and traditional approach would be Fisher's exact test.

Wiki should allow you to calculate it manually for your data.

If using R you could do something like this:

> cm1 <- matrix(c(53, 67, 29, 38),
                nrow = 2,
                dimnames = list(c("first", "control"), c("hits", "misses"))) 
> addmargins(as.matrix(cm1), c(1,2))
        hits misses Sum
first     53     29  82
control   67     38 105
Sum      120     67 187
> fisher.test(cm1)

Gives p=1 meaning probability of your data given the null is very high (suggests that first group v. unlikely to be different from control).

You can also do

fisher.test(cm1, alternative="greater")


The null hypothesis is that these two samples come from the same population. The alternative is that the first comes from a different population which has more "hits" than the second.

... but it's still p=0.52.

Now there are other methods but most will accept that it's unlikely that there's any great difference between 53/82 = 65% and 67/105 = 64%

  • 1
    $\begingroup$ The two sample proportions test was quite similar, though I did it two tailed. $\endgroup$
    – Glen_b
    Dec 12 '13 at 10:28
  • 1
    $\begingroup$ Really nice answer from dardisco, I don't have enough reputation so I will leave the link to python reference here docs.scipy.org/doc/scipy/reference/generated/… $\endgroup$
    – paloman
    Mar 31 at 11:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.