2
$\begingroup$

I've read in various places (see e.g. the comments on this question) that the concept of Type 1 error is incompatible with Bayesian paradigms for hypothesis testing.

Why is that exactly? I can't seem to put the different pieces and definitions of what "Bayesian hypothesis testing" actually means in order to see how Type 1 error doesn't make sense in that paradigm.

EDIT: one of the answers below seems to imply that Bayesian methods do not concern themselves with whether a hypothesis is true or false, i.e. that Bayesians only deal with assigning probabilities to hypotheses, not taking actions.

But then what about Bayes' rules (i.e. decision rules that minimize the Bayes risk)? Bayes' rules still result in rejection (or non-rejection) of the null hypothesis. So it fair to say that making any decision based on a Bayes' rule is non-Bayesian at some level? I'm clearly misunderstanding something here.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
5
$\begingroup$

The Bayesian approach is based on determining the probability of a hypothesis with a model using an "a priori" probability that is then updated based on data. On the contrary, the classical hypothesis testing does not admit assigning a probability to the null hypothesis, but just either accepting or refusing it. The error-I type is the probability of wrongly refusing the null hypothesis when it is true. Thus, it is something completely different from the Bayesian logic (since probability is referred to making a mistake, not to the hypothesis itself).

EDIT: I stressed the fact the Bayesian approach is based on assigning a probability to a hypothesis, because this is a crucial difference wrt the classical approach, that mantains parameters are "assigned by Nature", thus not random variables, so you can't make probability statements directly on them. However, after you get your a posteriori probability, then of course you can take action, either by chosing the hypothesis with higher probability, or the one minimizing a given cost function. See, for example, here: https://www.probabilitycourse.com/chapter9/9_1_8_bayesian_hypothesis_testing.php

To sum up, I'd say the difference is: "a posteriori probability vs p-value", not "making vs not making a decision".

Improve this answer
$\endgroup$
3
  • $\begingroup$ I'm still a bit confused. Then is it fair to say that it's non-Bayesian to use any Bayesian rule to make an accept/reject decision for a pair of hypotheses? I've edit my question above to reflect this. $\endgroup$
    – gogurt
    Commented Feb 27, 2018 at 22:49
  • $\begingroup$ I also edited my reply to clarify the issue. I didn't imply that the Bayesian approach does not lead to a decision. $\endgroup$
    – Federico Tedeschi
    Commented Feb 28, 2018 at 8:15
  • $\begingroup$ Note: You don't "accept" a null hypothesis in a hypothesis test, you either reject the null hypothesis or you don't reject the null hypothesis. Failing to reject the null hypothesis is not the same as accepting it. $\endgroup$
    – Javid Jamae
    Commented Aug 21, 2020 at 19:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.