1
$\begingroup$

Suppose we are given:
P(reject H0 | H0 is true) = $\alpha$ (probability of type I error)
P(don't reject H0 | H1 is true) = $\beta$ (probability of type II error)

Assume that we know the power of the test to be
P(reject H0 | H1 is true) = 0.99

Let's say that we didn't reject H0. Can we say anything about the probability of H0 being true in such case? Since H0 and H1 are mutually exclusive, and in the case of not rejecting H0, the probability of H1 is very small (since the power of the test is very large), can we say with 0.99 (99%) probability that H0 is true? Or taking into account the possibility of type I error (with probability $\alpha$), can we say with probability $0.99-\alpha$ that H0 is true?

Or if the previous is incorrent, is it possible to calculate the probability
P(H0 is true | we didn't reject H0) = ??

Using Bayes formula gives
P(H0 is true | we didn't reject H0) = P(we didn't reject H0 | H0 is true) * (P(H0 is true)/P(we didn't reject H0)).

The probability of complementary event P(¬A|B) = 1 - P(A|B)
So the likelihood on the right side of the bayes formula
P(don't reject H0 | H0 is true) = 1 - P(reject H0 | H0 is true) = $1- \alpha$.

Substituting this into right side of the Bayes formula we get
P(H0 is true | we didn't reject H0) = ($1-\alpha$) * (P(H0 is true)/P(we didn't reject H0)).
Can we say anything about P(H0 is true) ja P(we didn't reject H0)?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ You can't apply Bayes' formula until you have provided a prior distribution on the hypothesis space. $\endgroup$
    – whuber ♦
    Commented Feb 11, 2021 at 20:53
  • $\begingroup$ Are there recommandations/research on the types of priors one should set to transform hypothesis testing into the probability of the hypothesis given the observed? $\endgroup$
    – NoVariation
    Commented Feb 12, 2021 at 7:56

1 Answer 1

Reset to default
2
$\begingroup$

(1) H0 is a formal model and (as far as I think at least, and I'm not alone) no formal model will ever be true in reality, so the probability that H0 is true will be zero whatever the test outcome is. (Tests don't say anything about H0 being true, all they can say is whether the data are compatible with H0.)

(2) Within a Bayesian model (which is different from "in reality") you can have a probability that H0 is true, but this requires you to set up a prior distribution, including a prior probability that H0 is true. So you need to specify P(H0 is true) before data, then you also can compute its probability conditionally given the data ("posterior probability").

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.