# Type I error and type II error trade off

Can someone illustrate the trade off with the help of a diagram and intuitive explanation? Thanks

• Are you familiar with the story of the boy who cried wolf? May 10 '16 at 1:50
• I am! I read what type 1 error and type 2 error is in the story's context. May 10 '16 at 1:53

When the boy first pretends there is a wolf and the villagers believe him, it's a Type I error. When he claims there is a wolf again, but no one takes him seriously, though it is true, it's a type II error.

The villagers can avoid type I errors by never believing the boy, but that will always cause a Type II errors when there is a wolf around. Similarly, they can always believe him and never make a Type II, but that will cause lots of Type I errors.

You can think of using how scared the boy as a kind of test statistic. If he's crying, his breathing and heart rate are elevated, he has goose bumps or piloerection (his hair is standing up), then the villagers should take his claim more seriously. Requiring all these symptoms to be present and high is analogous to using a small $\alpha$ in the graph that @slowloris posted.

• That makes sense! Thanks a lot :) But could you illustrate that with a figure? I remember my professor showing me something in class about that. There were bell curves under null and alternative and we could see the trade off between type 1 and type 2 errors. May 10 '16 at 2:04

A life and death example of statistical errors

You are a paramedic and you approach the scene of a car accident. One victim is laying motionless on the road and you must assess whether the victim is dead or alive, and the victim will be treated accordingly. Based on this information which error rate results in the most costly mistake?

Null Hypothesis - The victim's status equals a living person
Alternative Hypothesis - The victim's status is not equivalent to a living person (i.e., they are dead)

Type I error - You reject the null hypothesis when the null hypothesis is actually true.
Type II error - You fail to reject the null hypothesis when the the alternative hypothesis is true.

Cost of Type I error - You erroneously presume that the victim is dead, and they do not receive an ambulance to the hospital for a life saving medical treatment.
Cost of Type II error - You erroneously send a dead person to the hospital in an ambulance.

Answer: As you can see, the Cost of the Type I error is tremendously worse than the cost of the Type II error.

Therefore, you may consider the trade-off of these errors. In traditional statistical hypothesis testing, you could use this cost benefit analysis to determine your alpha (Type I error rate) and beta (Type II error rate) before conducting an experiment. In our example, we would want a very small alpha (typically 0.05), but could live with a larger beta (typically 0.1 - 0.3, but we could live with >0.3 in this case). This would, for example, dramatically effect the required sample size of your experiment because you are OK with accepting the null hypothesis incorrectly and report that dead people are actually alive.

I diagnostic testing, we can look at trade off of Type I and Type II errors in terms of the threshold we place between the distribution of a measurement taken from two populations. Using our example, we might measure the redness of the skin, with redder skin representing a living victim.

In the figure, we can see that the best place to put a threshold between these groups is in the lowest point between the two distributions. This location would result in the least overall error. However, we can make a logical trade off here: By moving the threshold to the Right, the probability of a Type I error is reduced at the expense of increasing the probability of a Type II error. In our example, this trade off is good and would likely save someone's live and our job as a paramedic.

Google "illustration type 1 type 2 error" and have your pick. This site is one of many you can find a figure like the one you are asking for.

• Thanks :) this is exactly what I think I saw in my class. May 10 '16 at 2:18