0
$\begingroup$

I am new to the Bayesian world, and I'm trying to understand how hypotheses tests are performed here (as opposed to the frequentist framework).

I am aware that likelihoods, priors and posteriors can be discrete or continuous. And once we have calculated posteriors, we can build a lot of things like credible intervals and so on.

Now, the problem arises when I'm applying this to hypotheses tests. So far I encountered situations where there were a finite number of hypotheses to compare ($H_0$, $H_1$, $H_2$..), as well as a discrete number of parameters associated to them.

For example, I'm tossing a coin n times with associated observations $X_1, ..., X_n$ where: $$X_i \sim Bernoulli(\theta)$$ where $\theta$ is the probability of getting heads.

I want to test whether this coin is loaded or not and I may have prior beliefs such that: \begin{cases} p(\theta=0.5) = 0.5\\ p(\theta=0.7) = 0.5\end{cases}

I would then have my hypotheses $H_0$ (coin is fair) and $H_1$ (coin is loaded).
I would then calculate a posterior for each hypothesis and conclude.

But what if $\theta$ was continuous (e.g. follows a Uniform distribution) ? What would my hypotheses be ? And how to calculate them ?

$\endgroup$
3
$\begingroup$

The case is well-covered in Bayesian textbooks, including ours!, and can be summarised by the constraint that one can only test hypotheses for which the prior has a positive probability mass. When the prior is given as a Uniform(0,1), it is impossible to test whether or not $\theta=1/2$. A contrario, if one comes up with the question as to whether or not $\theta=1/2$, a prior must be constructed with this possibility in mind, hence must include a point mass at $1/2$. For instance, a mixture of a Uniform(0,1) and of a point mass at $1/2$.

$\endgroup$
1
  • $\begingroup$ thanks for your answer ! So to understand it better, if I take your example of a uniform prior with a point mass at 1/2, what would my hypotheses be? Would the posterior for $H_0$ be constructed for a $\theta = 1/2$ (i.e. a discrete prior), while the posterior for $H_1$ would integrate $\theta$ over [0,1] (i.e. continuous prior) ? @Xi'an $\endgroup$
    – Derbs
    Aug 10 '19 at 0:07
3
$\begingroup$

Contrary to what Fabrice is saying, mixed discrete-continuous distributions are completely fine mathematically. His comment about Lebesgue-negligable sets and the contradiction doesn't really make sense--in first case, the set ${0.5}$ is, of course measure 0 under the measure associated with any continuous random variable. However, changing the distribution of the prior changes the measure, and more importantly, linear combinations of measures are fine. We are just not longer using only Lebesgue measure to compute probabilities.

Further, this sort of statement is completely plausible(though a bit contrived) in real life--imagine you have two big bags of coins. One is comprised completely of fair coins, and the other has some continuous distribution of giving heads. Dump these two together, and you will obtain the case that Xi'an described.

Having a mixed distribution even occurs in finance with option returns. If you pay x for your contracts, Your return is -x if the contracts expire with the share price below the strike. The share price has positive probability of being below the strike at expiry, so there is a probability mass of positive measure at -x when looking at the distribution of option returns.

In the case that you mention in your comment, I'll interpret the problem as being that $H_0$ is that p=0.5, and that $H_1$ is that $p \sim P$, where the pdf of P is $f_p (x)$ Then the Bayes factor $H_1/H_0$ given k successes in n trials will be $$\frac{\int_0 ^1 {n \choose k}(x^k)(1-x)^{n-k}f_p (x)dx}{{n \choose k}(0.5^k)(0.5)^{n-k}}$$

$\endgroup$
2
  • $\begingroup$ Thanks for your answer! If my prior distribution was uniform on [0,1] with a point mass of 0.5 at $\frac {1}{2}$, would $f_p(x)$ in your expression simply be 0.5 ? Also, glad you brought Bayes factors in here as I have a question on that: to decide between hypotheses one could calculate a posterior distribution (which accounts for both likelihood and prior) for each hypothesis, and select the one with the highest posterior probability. But Bayes factor is simply a ratio of likelihoods, and is not dependent on priors. How can this be a good indicator for a bayesian hypothesis testing? @Alex $\endgroup$
    – Derbs
    Aug 12 '19 at 18:47
  • $\begingroup$ To illustrate this: we know that $$\frac{p(H_1\mid data)}{p(H_0\mid data)} = BayesFactor * \frac{p(H_1)}{p(H_0)}$$ where $$BayesFactor = \frac{p(data\mid H_1)}{p(data\mid H_0)}$$ Now no matter what prior beliefs you have ($p(H_1) = 0.5$ and $p(H_0) = 0.5$, or $p(H_1) = 0.7$ and $p(H_1) = 0.3$, or whatever), the Bayes Factor doesn't change. Am I wrong ? Why would we use that to determine which hypothesis to choose within a Bayesian framework, where we usually take both prior and likelihood to make decisions ? @Alex $\endgroup$
    – Derbs
    Aug 12 '19 at 18:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.