1
$\begingroup$

I've got to teach a basic course on the $\chi^2$ test soon, and I will of course insist on the fact that $p$ is not the probability of the null hypothesis to be true, and that the result of the test can't allow to accept the null. However, students of this course won't have any mathematical or statistical background, and I'd like to provide an explanation as intuitive as possible.

I've read some interesting things in this ScienceNews article, with an example of a dog barking when he's hungry, but I still don't find this really clear enough.

Did you already provide such an explanation, or have any idea or example that would be "sufficiently intuitive" ?

Thanks a lot in advance.

|cite|improve this question
$\endgroup$
  • $\begingroup$ Already: here, here, here, & probably elsewhere. $\endgroup$ – Scortchi - Reinstate Monica♦ Jul 3 '13 at 17:28
  • 1
    $\begingroup$ I think there are a couple of threads on CV that might help you; take a look at these: Why does a 95% CI not imply a 95% chance of containing the mean?, Understanding p-value, & Interpretation of p-value in hypothesis testing. $\endgroup$ – gung - Reinstate Monica♦ Jul 3 '13 at 17:30
  • $\begingroup$ I don't think there will be a good intuitive explanation of why the p-value is not the probability that the null hypothesis is true because the principal reason this isn't true (that frequentist statistics cannot assign a probability to the truth of a hypothesis as it has no valid interpretation as a long run frequency) is deeply counter-intuitive. $\endgroup$ – Dikran Marsupial Jul 3 '13 at 17:46
  • 2
    $\begingroup$ For what it's worth, I don't think this is an exact duplicate. You might try editing this in a way to make 'looking for teaching strategies for making the fact that $p(d|H_0)\ne p(H_0|d)$ intuitive' more distinct from the prior questions, & maybe it can be reopened. $\endgroup$ – gung - Reinstate Monica♦ Jul 3 '13 at 18:20
  • 1
    $\begingroup$ @gung I would add my vote to reopen if the focus were on teaching strategies (instead of "explanation") and specifically on the "$p\ne$ probability of null" question, because those elements could lead to a different set of answers. $\endgroup$ – whuber♦ Jul 3 '13 at 19:11
3
$\begingroup$

Here is an example that I use for teaching. I take a real coin and toss it three times. Every time I get the same result. We go through the calculations: we find that the probability that we get three tails in three throws under H0 is $\frac{1}{8} = 0.125 > 0.05$. Thus, we cannot reject the H0.

Then I continue to toss the coin, and every time I am getting exactly the same result. The students (without doing any calculations) see that the coin (in fact, my tossing procedure) is rigged. We reject the H0. The H0 never was true; we just did not have enough data to demonstrate that. Also, we see that the calculations of the p value only involve the parameter for a fair coin and a fair toss.

I also use the same example to demonstrate a few other pitfalls. For example, the students usually assume that the coin is rigged; the fact that it is not serves as a metaphor of a flawed measurement procedure.

Here is how to get the same result every time: http://www.youtube.com/watch?v=jQOX7Gwz69U

(note that I do not teach statistics for living; rather, it is a part of a bioinformatic course)

|cite|improve this answer
$\endgroup$
  • $\begingroup$ What about the test statistic? After all the probability under the null of heads, followed by tails, then by heads - indeed of any sequence - is also 0.125 $\endgroup$ – Scortchi - Reinstate Monica♦ Jul 3 '13 at 17:40
  • $\begingroup$ You are right, I should have not written "consecutive" $\endgroup$ – January Jul 3 '13 at 18:44
  • $\begingroup$ I do not want to twist this answer, but it seems that the demonstration could be misconstrued as showing that asymptotically, a sufficiently large experiment will produce a p-value that is approximately the same as the "probability of $H_0$." In other words, this (nice) demonstration might not accomplish the learning objectives you believe it does, because it does not appear directly to attack the fundamental misconception (in the frequentist paradigm, of course) that it even makes sense to assign a probability to the null. $\endgroup$ – whuber♦ Jul 3 '13 at 19:14

Not the answer you're looking for? Browse other questions tagged or ask your own question.