It's very easy to find famous examples of Type I errors in the histories of various fields. However, I'm struggling to find even one clear example of a Type II error. For my purposes, I'm specifically looking for an example that:

  • can be easily described to an undergraduate class in a few minutes or less
  • actually occurred (rather than being hypothetical)
  • 1
    $\begingroup$ As for your first bullet, have a look here: stats.stackexchange.com/questions/110433/… $\endgroup$ – Stefan Jan 9 '17 at 18:36
  • $\begingroup$ I looked at Stefan's link. There are indeed examples that are easy to explain. Special attention is paid to Alexis picture of diagnosing pregnancy. There are only hypothetical examples but I am sure you can find real one's in the literature. Someone should be able to come up with something for you. $\endgroup$ – Michael R. Chernick Jan 9 '17 at 19:13
  • $\begingroup$ Do you mean something like "smoking does not cause cancer"? $\endgroup$ – whuber Jan 9 '17 at 19:19
  • 2
    $\begingroup$ That might work, assuming the study was conducted in good faith. I wouldn't want students to confuse "failing to detect an effect" with "intentionally hiding an effect." $\endgroup$ – Anthony Jan 9 '17 at 19:25
  • 1
    $\begingroup$ What about some of standard examples typically cited for the utility of meta analysis? E.g. sudden infant death syndrome or beta blockers for mi. $\endgroup$ – Björn Jan 9 '17 at 21:51

"The harm done by tests of significance" (pdf) relates 3 true stories in which not rejecting the null while it was actually false (so making a Type II error) has been interpreted as accepting the null and had lead to bad general interest decisions. Here is a brief summary of one of them:

  • The practice of allowing right-turn-on-red (or RTOR) at signalized intersections started in California in 1937.

  • To see weither this can be generalized to other state, a consultant did before–after study at 20 intersections for Virginia and concluded, quite correctly, that the change was not statistically significant.

  • More published studies followed. An example: one study in 1977 found that there were 19 crashes involving right turning vehicles before and 24 after allowing RTOR and "conclude correctly that “this increase in accidents in not statistically significant, and therefore it cannot be said that this increase in RTOR accidents is attributable to RTOR”". Several small studies all pointing in the same direction get published but with statistically not significant results continued to accumulate all concluding that there was no significant difference in crashes.

  • "After RTOR became nearly universally used in North America, several large data sets became available and the adverse effect of RTOR could be established."

The author concludes:

Researchers obtain real data which, while noisy, time and again point in a certain direction. However, instead of saying: “here is my estimate of the safety effect, here is its precision, and this is how what I found relates to previous findings”, the data is processed by NHST, and the researcher says, correctly but pointlessly: “I cannot be sure that the safety effect is not zero”.

  • $\begingroup$ I am a little confused. It seems like the standard null and alternative hypotheses have been switched. Is this what the OP intended. What might normally be considered a type I error becomes a type II error becuse the null and alternative are switched, $\endgroup$ – Michael R. Chernick Jan 9 '17 at 19:53
  • $\begingroup$ @MichaelChernick Sorry I don't get it. In my understanding, the null is "right-turn-on-red is similar to the standard rule", not enough evidence against this null leads to a Type II error. Are you okay with that? $\endgroup$ – peuhp Jan 9 '17 at 20:09
  • $\begingroup$ I accept your point. Here is a different issue. A type II error is failure to reject a false null hypothesis. The original study failed to reject it but to be a type ii error you need to know the null hypothesis is false. If the study in 1937 failed to reject the null hypothesis how does a study many years later confirm that the null hypothesis was "true". It is an entirely different population of drivers with different cars. I think this may be a problem for the speed limit change example also. $\endgroup$ – Michael R. Chernick Jan 9 '17 at 20:30
  • 1
    $\begingroup$ @MichaelChernick This is indeed a point to discuss but that IMHO does not diminish the impact of the examples discussed in the paper. $\endgroup$ – peuhp Jan 10 '17 at 9:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.