# H to check if proportions in two small samples are the same

I have two small samples denoting if something occured or not. For example:

$s_1=\left(1,1,1,0,0,1\right)$

$s_2=\left(1,0,0,0,0,1,1,0\right)$

I want to test if proportions of occuring $1$ are the same. I cannot use proportion test for two samples, as samples are too small. Can I use Chi square homogenity test and get the robust outcome?

• Of course you can use a test of proportions on these data. Your real problem is that you haven't enough data to discern any but the very largest differences between proportions. Unfortunately--in this case--that problem will not go away by choosing another test. You have only three kinds of options: (1) make some strong assumptions (such as a prior distribution); (2) collect more data; (3) accept the results of the test of proportions (which would be not to reject the null hypothesis of equal proportions). Which of these options are ones you would consider? – whuber Jan 2 '15 at 15:38
• Take a look at fischers exact test. Also, you can find a calculator on the net to do the calculations – user64985 Jan 6 '15 at 8:09

• Unfortunately, exact implementations of tests of proportions will not make these data appear significantly different (at least for any reasonably small test size). For instance, even an exact one-sided test has a p-value of $\sum_{i=4}^{4+3}\binom{6}{i}\binom{8}{4+3-i}/\binom{6+8}{4+3}=1016/3432\approx 30\%$. – whuber Jan 2 '15 at 16:26
• No, that's exactly my point: there is not enough evidence here. To obtain a significant result (at $\alpha=0.05$) with two samples of sizes six and eight, conditional on observing seven positives, you would need all elements of the smaller sample to be positive. And you could determine that using the standard test of proportions (but, as you point out, only by applying the exact sampling distribution, because an asymptotic approximation would be suspect). – whuber Jan 2 '15 at 17:02