0
$\begingroup$

Suppose I am in the rare situation where I know for certain the prior probability of the null hypothesis. And yes, I mean the marginal probability of the null (not the probability of the null, conditional on the data). Further suppose that I am comparing two groups (i.e., a t-test) and already know the prior probability that my treatment (whatever it is) will work.

Now suppose I want to enter that information into some statistical software that uses Bayesian statistics. To my utter dismay, what do I find but that the software requires me to specify the prior distribution of the parameters, not the null itself.

That is my dilemma.

So, how do I convert a prior probability (in probability metric) into parameter means/standard deviation distributions?

Put differently, given a prior (say p = 0.5 that the treatment "works"), what should the distribution of $\mu_1, \mu_2, \sigma_1, \text{ and } \sigma_2$ look like?

$\endgroup$
  • $\begingroup$ You don't seem to have a full prior probability to specify, because your probabilities do not integrate to unity: you have to give a distribution for the alternative hypothesis as well. $\endgroup$ – whuber Sep 7 '18 at 21:43
  • $\begingroup$ @whuber: let us assume the distributions are mutually exclusive (so, p[alternative] = 1-p[null]). $\endgroup$ – dfife Sep 10 '18 at 20:10
  • $\begingroup$ In a t-test the alternative is an entire continuum of possibilities, not just one. You have to give a distribution over this entire set of possibilities. A single number won't do. $\endgroup$ – whuber Sep 10 '18 at 20:23
1
$\begingroup$

If your null hypothesis is that the treatments are exactly equal (i.e. not just really close, but absolutely completely equal), then parameterizing your model in terms of mean control group outcome and treatment difference $\delta$ makes sense. You can then assign a prior with a point mass at zero (with weight equivalent too the prior probability of the null hypothesis) to the treatment difference to the treatment difference with the non- discrete part of the prior reflecting your best guess as to the treatment difference when there is one. Be aware that this can run into Lindley's paradox.

If your null hypothesis is about an irrelevant difference such as $\delta \in [-\epsilon, \epsilon]$, then some continuous distribution that puts enough prior mass into this interval is an option.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.