0
$\begingroup$

I am new to the Bayesian world, and I'm trying to understand how hypotheses tests are performed here (as opposed to the frequentist framework).

I am aware that likelihoods, priors and posteriors can be discrete or continuous. And once we have calculated posteriors, we can build a lot of things like credible intervals and so on.

Now, the problem arises when I'm applying this to hypotheses tests. So far I encountered situations where there were a finite number of hypotheses to compare ($H_0$, $H_1$, $H_2$..), as well as a discrete number of parameters associated to them.

For example, I'm tossing a coin n times with associated observations $X_1, ..., X_n$ where: $$X_i \sim Bernoulli(\theta)$$ where $\theta$ is the probability of getting heads.

I want to test whether this coin is loaded or not and I may have prior beliefs such that: \begin{cases} p(\theta=0.5) = 0.5\\ p(\theta=0.7) = 0.5\end{cases}

I would then have my hypotheses $H_0$ (coin is fair) and $H_1$ (coin is loaded).
I would then calculate a posterior for each hypothesis and conclude.

But what if $\theta$ was continuous (e.g. follows a Uniform distribution) ? What would my hypotheses be ? And how to calculate them ?

$\endgroup$
3
$\begingroup$

The case is well-covered in Bayesian textbooks, including ours!, and can be summarised by the constraint that one can only test hypotheses for which the prior has a positive probability mass. When the prior is given as a Uniform(0,1), it is impossible to test whether or not $\theta=1/2$. A contrario, if one comes up with the question as to whether or not $\theta=1/2$, a prior must be constructed with this possibility in mind, hence must include a point mass at $1/2$. For instance, a mixture of a Uniform(0,1) and of a point mass at $1/2$.

$\endgroup$
  • $\begingroup$ thanks for your answer ! So to understand it better, if I take your example of a uniform prior with a point mass at 1/2, what would my hypotheses be? Would the posterior for $H_0$ be constructed for a $\theta = 1/2$ (i.e. a discrete prior), while the posterior for $H_1$ would integrate $\theta$ over [0,1] (i.e. continuous prior) ? @Xi'an $\endgroup$ – Derbs Aug 10 '19 at 0:07
3
$\begingroup$

Contrary to what Fabrice is saying, mixed discrete-continuous distributions are completely fine mathematically. His comment about Lebesgue-negligable sets and the contradiction doesn't really make sense--in first case, the set ${0.5}$ is, of course measure 0 under the measure associated with any continuous random variable. However, changing the distribution of the prior changes the measure, and more importantly, linear combinations of measures are fine. We are just not longer using only Lebesgue measure to compute probabilities.

Further, this sort of statement is completely plausible(though a bit contrived) in real life--imagine you have two big bags of coins. One is comprised completely of fair coins, and the other has some continuous distribution of giving heads. Dump these two together, and you will obtain the case that Xi'an described.

Having a mixed distribution even occurs in finance with option returns. If you pay x for your contracts, Your return is -x if the contracts expire with the share price below the strike. The share price has positive probability of being below the strike at expiry, so there is a probability mass of positive measure at -x when looking at the distribution of option returns.

In the case that you mention in your comment, I'll interpret the problem as being that $H_0$ is that p=0.5, and that $H_1$ is that $p \sim P$, where the pdf of P is $f_p (x)$ Then the Bayes factor $H_1/H_0$ given k successes in n trials will be $$\frac{\int_0 ^1 {n \choose k}(x^k)(1-x)^{n-k}f_p (x)dx}{{n \choose k}(0.5^k)(0.5)^{n-k}}$$

$\endgroup$
  • $\begingroup$ Thanks for your answer! If my prior distribution was uniform on [0,1] with a point mass of 0.5 at $\frac {1}{2}$, would $f_p(x)$ in your expression simply be 0.5 ? Also, glad you brought Bayes factors in here as I have a question on that: to decide between hypotheses one could calculate a posterior distribution (which accounts for both likelihood and prior) for each hypothesis, and select the one with the highest posterior probability. But Bayes factor is simply a ratio of likelihoods, and is not dependent on priors. How can this be a good indicator for a bayesian hypothesis testing? @Alex $\endgroup$ – Derbs Aug 12 '19 at 18:47
  • $\begingroup$ To illustrate this: we know that $$\frac{p(H_1\mid data)}{p(H_0\mid data)} = BayesFactor * \frac{p(H_1)}{p(H_0)}$$ where $$BayesFactor = \frac{p(data\mid H_1)}{p(data\mid H_0)}$$ Now no matter what prior beliefs you have ($p(H_1) = 0.5$ and $p(H_0) = 0.5$, or $p(H_1) = 0.7$ and $p(H_1) = 0.3$, or whatever), the Bayes Factor doesn't change. Am I wrong ? Why would we use that to determine which hypothesis to choose within a Bayesian framework, where we usually take both prior and likelihood to make decisions ? @Alex $\endgroup$ – Derbs Aug 12 '19 at 18:48
0
$\begingroup$

This is a quick reply to Alex's answer above. I can't comment because I'm out of credits!

I fear you miss the point.

I wrote: "such an awful mixture... for a single coin.".

I'm definitely not talking about the existence of mixed discrete-continuous distributions, which is obvious. If you mix anything and everything (e.g. your bags of coins), then of course you get mixtures of anything and everything. It follows that your examples are irrelevant.

I'm talking about what Xi'an calls (see The Bayesian Choice) the "measure-theoretic puzzling modification of continuous [and natural] probability distributions into mixed discrete-continuous [and not so natural] distributions" that arises in testing point null hypotheses.

It is therefore your duty to solve this puzzle because, you know, what we need are not puzzles but theories and theorems. You need to go through two steps.

Step 1: answer the question: is this puzzling modification internal or external to Boolean algebra and probability theory? If it is internal, then you can derive such a modification from the point null hypothesis, seen here as a new Boolean proposition added to the prior Keynes-Borel-Cox-Jaynes-Bernardo corpus of knowledge. I tried to do this 15 years ago (in R. T. Cox style) but I failed and I'm not aware of any result in this direction. Are you? If you can't derive this modification, you are in a weak position because you make a call to something which is external to probability theory.

Step 2: it is also your duty to tell us how to properly assign prior probabilities to such mixed hypotheses (e.g. discrete H0 and continuous H1 almost everywhere). I sincerly wish you good luck because the modification has introduced a monstrous asymmetry between H0 and H1 while probability theory basically relies on symmetry and invariance. For instance, I often see p(theta=theta0) = p(theta <> theta0) = 1/2 which is an obviously illegal and provably wrong application of the principle of insufficient reason. Concretely, please tell us what p(theta=theta0) precisely is if it's no longer equal to 0...

If you can't go through those steps, then you may like to consider the alternative way I mentioned, which is internal to probabilty theory. You don't need any modification of anything, just to forget actual infinity and go back to potential infinity.

Finally, your last quantity is a genuine Bayes factor only if said modification is internal to probability theory (otherwise the genuine Bayes factor is undefined).

$\endgroup$
-1
$\begingroup$

Xi’an answer above is plain wrong and insane.

If $\theta$ is a continuous parameter, then a basic theorem of measure theory (a singleton has Lebesgue measure $0$, it is said to be Lebesgue-negligible.) tells us that the probability that $\theta$ is equal to any particular value $\theta_0$ (e.g. $\theta_0 = 1/2$) is equal to $0$, a priori and a posteriori. Hence, it is not impossible at all to test whether or not $\theta = \theta_0$: the answer is simply “NO, it isn’t!” and you don’t need any data to answer this question. First mistake in Xi’an answer.

Second, assigning a non-zero probability, a point mass to such Lebesgue-negligible set as proposed by Xi’an just amounts to saying that a Lebesgue-negligible set is no more Lebesgue-negligible! That’s insane, it’s nothing but a contradiction, isn’t it? The insanity comes from the fact that he tries to overcome the negligibility issue after its appearance. But it is too late: of course, the right way is to treat and to solve an issue before it occurs, not after! And ask yourself how you could ever come up with such an awful mixture of a continuous distribution (almost everywhere) and a discrete/point mass distribution… for a single coin??? It simply makes no sense.

A much more meaningful question with a straightforward answer, to be recommended for beginners like you, is simply to test the continuum of hypotheses

$H_{\theta_0} : \theta = \theta_0$

which is nothing but estimating the parameter $\theta$. This is done by computing its posterior probability distribution as usual by Bayes rule. If this distribution concentrates its mass away for ½, then you can suspect the coin (or coin tosses) to be unbalanced.

But after all, there might exist a non-trivial and non-insane answer to the original question/test $\theta = \theta_0$. It is obtained by working before the Lebesgue-negligibility issue occurs.

Clearly, this question has a non-trivial and non-insane answer if the parameter $\theta$ is discrete, which is basically given by the Bayes factor between the null hypothesis and its negation. So, the idea it just to discretize the continuous parameter $\theta$ over some partition/grid with some finite step, compute said Bayes factor for the discrete random variable and takes the limit when the step goes to zero. Very simple and not necessarily insane. Unfortunately, to the best of my knowledge, you won’t find this solution in any textbook. In particular, it has not been proved wrong yet but I can assure you that it works well in practice!

Finally, you may wonder how is it possible for a question to have two different answers, a trivial one and a non-trivial one? Just because the trivial answer is absolute while the non-trivial one is relative.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.