# Appropriate test for comparing two sets of binary tests?

I have items made from two different materials. For each material I took a set of 250 samples and subjected them to a test which either broke them or did not.

Which test could answer the question - is there a significant difference between the breakage rates of the two materials. I constructed a contingency table but am not sure how to proceed - some sort of chi-squared test?

$$\begin{matrix} & \text{Broken} & \text{Unbroken}\\ \text{Material A} & 2 & 248 \\ \text{Material B} & 1 & 249\\ \end{matrix}$$

The numbers of broken samples was small, as shown in the table.

• Based on these data, if some statistical test showed that material A and material B were different, would you believe it?
– John
Dec 17 '12 at 17:40

• +1 One way to provide intuition concerning your last remark is to point out that under the null hypothesis (of equal breakage rates), we may view the distribution of the three broken samples among the two rows (materials) as being a matter of pure chance, as if a fair coin had been flipped for each breakage to determine which row it would be assigned to. We would see some imbalance in the assignments (because $3$ is odd) and even the most extreme imbalance where all breakages occur for a single material would have a chance of $1/8+1/8=1/4$, too big to be considered "significant."