Intuitive explanation of why p is not the probability of the null hypothesis I've got to teach a basic course on the $\chi^2$ test soon, and I will of course insist on the fact that $p$ is not the probability of the null hypothesis to be true, and that the result of the test can't allow to accept the null. However, students of this course won't have any mathematical or statistical background, and I'd like to provide an explanation as intuitive as possible.
I've read some interesting things in this ScienceNews article, with an example of a dog barking when he's hungry, but I still don't find this really clear enough.
Did you already provide such an explanation, or have any idea or example that would be "sufficiently intuitive" ?
Thanks a lot in advance.
 A: Here is an example that I use for teaching. I take a real coin and toss it three times. Every time I get the same result. We go through the calculations: we find that the probability that we get three tails in three throws under H0 is $\frac{1}{8} = 0.125 > 0.05$. Thus, we cannot reject the H0.
Then I continue to toss the coin, and every time I am getting exactly the same result. The students (without doing any calculations) see that the coin (in fact, my tossing procedure) is rigged. We reject the H0. The H0 never was true; we just did not have enough data to demonstrate that. Also, we see that the calculations of the p value only involve the parameter for a fair coin and a fair toss.
I also use the same example to demonstrate a few other pitfalls. For example, the students usually assume that the coin is rigged; the fact that it is not serves as a metaphor of a flawed measurement procedure.
Here is how to get the same result every time: http://www.youtube.com/watch?v=jQOX7Gwz69U
(note that I do not teach statistics for living; rather, it is a part of a bioinformatic course)
