# Can you create an example showing when the p-value $P(D|H_0)$ does not imply probability of $H_0$ being true given the observed data $P(H_0|D)$?

I'm reading The Insignificance of Null Hypothesis Significance Testing and on page 654, the author states that most people incorrectly think that the null hypothesis significance test produces $$\mathbb{P}(H_0|D)$$: the probability of $$H_0$$ being true given the observed data.

But the test actually produces $$\mathbb{P}(D|H_0)$$. And by Bayes law, these two are not the same unless $$\mathbb{P}(H_0) = \mathbb{P}(D)$$. What is the intuitive meaning of this result?

Can someone give me an example where using typical hypothesis i.e. $$H_0: \mu = 0$$ and $$H_1: \mu \ne 0$$, $$\bar{X}$$ (sample mean) test statistic and the normal distribution, where hypothesis tests start to fail?

Is not possible to create the example that you looking for. The problem here is that $$P(H_0|D)$$ is a meaningless writing. The last because, even if in practice case you do not known if $$H_0$$ is true or false, we have to remember that, under the paradigm behind the p-value (ML and/or LS and then frequentist approach) parameters are unknown constants and not random variables.

Then in your example $$\mu$$ is a fixed constant (known or unknown) and something like $$P(-x<\mu is a nonsense writing. As a consequence even $$P(H_0|D)$$ is senseless. At the other side is possible to shown that p-value=$$P(D|H_0)$$ make sense.

Therefore, even if it seems an intuitive way, is not possible to use the bayes rule here in order to link $$P(D|H_0)$$ and $$P(H_0|D)$$

Therefore

But the test actually produces $$P(D|H_0)$$. And by Bayes law, these two are not the same unless $$P(H_0)=P(D)$$. What is the intuitive meaning of this result?

It haven't proper meaning.

• I absolutely agree with you. When I tried to reconstruct the same I had the exact problem with the $\mu$ part. Does it mean the way the author constructed his criticism is wrong? Sep 17, 2020 at 12:41
• I do not checked there, however note that the notion of p-value have produced a notably amount of missunderstandigs. Sep 17, 2020 at 13:15