I have items made from two different materials. For each material I took a set of 250 samples and subjected them to a test which either broke them or did not.

Which test could answer the question - is there a significant difference between the breakage rates of the two materials. I constructed a contingency table but am not sure how to proceed - some sort of chi-squared test?

$$ \begin{matrix} & \text{Broken} & \text{Unbroken}\\ \text{Material A} & 2 & 248 \\ \text{Material B} & 1 & 249\\ \end{matrix} $$

The numbers of broken samples was small, as shown in the table.

  • 2
    $\begingroup$ Based on these data, if some statistical test showed that material A and material B were different, would you believe it? $\endgroup$
    – John
    Dec 17 '12 at 17:40

Pearson's chi-squared test for association can be used for this sort of problem. For tables with low expected values, like your one, Fisher's Exact Test is better. But you don't need to do any statistical test to see that you need more data to tell whether A or B has the highest breakage rate.

  • 2
    $\begingroup$ +1 One way to provide intuition concerning your last remark is to point out that under the null hypothesis (of equal breakage rates), we may view the distribution of the three broken samples among the two rows (materials) as being a matter of pure chance, as if a fair coin had been flipped for each breakage to determine which row it would be assigned to. We would see some imbalance in the assignments (because $3$ is odd) and even the most extreme imbalance where all breakages occur for a single material would have a chance of $1/8+1/8=1/4$, too big to be considered "significant." $\endgroup$
    – whuber
    Dec 17 '12 at 17:09
  • 1
    $\begingroup$ Agreed. Now I know the appropriate test I can estimate how much data I would need to start to see differences. $\endgroup$
    – Peter Hull
    Dec 19 '12 at 9:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.