0
$\begingroup$

I'm reading The Insignificance of Null Hypothesis Significance Testing and on page 654, the author states that most people incorrectly think that the null hypothesis significance test produces $\mathbb{P}(H_0|D)$: the probability of $H_0$ being true given the observed data.

But the test actually produces $\mathbb{P}(D|H_0)$. And by Bayes law, these two are not the same unless $\mathbb{P}(H_0) = \mathbb{P}(D)$. What is the intuitive meaning of this result?

Can someone give me an example where using typical hypothesis i.e. $H_0: \mu = 0$ and $H_1: \mu \ne 0$, $\bar{X}$ (sample mean) test statistic and the normal distribution, where hypothesis tests start to fail?

$\endgroup$
0
$\begingroup$

Is not possible to create the example that you looking for. The problem here is that $P(H_0|D)$ is a meaningless writing. The last because, even if in practice case you do not known if $H_0$ is true or false, we have to remember that, under the paradigm behind the p-value (ML and/or LS and then frequentist approach) parameters are unknown constants and not random variables.

Then in your example $\mu$ is a fixed constant (known or unknown) and something like $P(-x<\mu<x)$ is a nonsense writing. As a consequence even $P(H_0|D)$ is senseless. At the other side is possible to shown that p-value=$P(D|H_0)$ make sense.

Therefore, even if it seems an intuitive way, is not possible to use the bayes rule here in order to link $P(D|H_0)$ and $P(H_0|D)$

Therefore

But the test actually produces $P(D|H_0)$. And by Bayes law, these two are not the same unless $P(H_0)=P(D)$. What is the intuitive meaning of this result?

It haven't proper meaning.

$\endgroup$
2
  • $\begingroup$ I absolutely agree with you. When I tried to reconstruct the same I had the exact problem with the $\mu$ part. Does it mean the way the author constructed his criticism is wrong? $\endgroup$ – Darby Bond Sep 17 '20 at 12:41
  • $\begingroup$ I do not checked there, however note that the notion of p-value have produced a notably amount of missunderstandigs. $\endgroup$ – markowitz Sep 17 '20 at 13:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.