I have a very simple question for Hypothesis testing. Is there a way to calculate the probability that the null hypothesis is false?

Since in the literature they're always talking about the significance level and the power of a test, these values are calculated assuming something about the null hypothesis (being true or false in each case).

But, in general terms, can we calculate the probability that a given null hypothesis is false, or true?

  • 1
    $\begingroup$ Well, maybe if you're Bayesian ... but people may not agree with your priors, on which such calculations would depend. $\endgroup$
    – Glen_b
    Aug 25, 2016 at 2:15
  • 1
    $\begingroup$ See stats.stackexchange.com/questions/166323/… $\endgroup$
    – user83346
    Aug 25, 2016 at 6:23
  • 1
    $\begingroup$ You can also check threads tagged tost $\endgroup$
    – Tim
    Aug 25, 2016 at 7:34

1 Answer 1


The term "null hypothesis" is usually used in a frequentist setting, where characteristics of the population, such as its mean, are regarded as fixed, not random. There, it makes no sense to talk about the probability of the null hypothesis.

In a Bayesian setting, these characteristics are regarded as random and we can talk about things like the probability of a population mean equalling 0. However, a typical Bayesian would give a prior probability of 0 to many common frequentist null hypotheses, such as the hypothesis that the mean of a normal distribution exactly equals a prespecified value.

  • 1
    $\begingroup$ So i can never measure the probability that a Null Hypothesis stated by someone is false right? All i can do (from the frequentist point of view) is Reject Ho if my data is telling me that. Can you give me a recomendation for an introductory Book of Bayesian Inference? Thanks a lot! $\endgroup$
    – S. Cow
    Aug 25, 2016 at 11:15
  • 1
    $\begingroup$ Yes and yes. Try John K. Kruschke's Doing Bayesian Data Analysis for a gentle introduction or Gelman et al.'s Bayesian Data Analysis for a deeper book that assumes some familiarity with mathematical statistics. $\endgroup$ Aug 25, 2016 at 14:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.