1
$\begingroup$

There's a lot to be said and read about this, but I haven't found a clear answer to this question:

Bayesian statistics are said to 'penalize' vague hypotheses with weak priors, by giving more support for the null hypothesis.
Say the theory I'm interested in proving actually predicts that the null is true. I could cheat, and set up my model with a weak prior. This would unfairly bias my empirical evidence towards the null. How can this be prevented?

Thanks!

EDIT: I realized I was actually referring to Lindley's paradox, where as far as I understand) a precise null and an uninformative prior might bias towards the null, whereas frequentist statistics would reject the null. http://www.laeuferpaar.de/Papers/LindleyPSA.pdf

$\endgroup$
  • $\begingroup$ What is your null hypothesis ? Something like $H_0\colon\{\theta=\theta_0\}$ ? $\endgroup$ – Stéphane Laurent Jul 22 '14 at 7:47
  • $\begingroup$ Yes, something like this. $\endgroup$ – elisa Jul 22 '14 at 8:22
  • $\begingroup$ Reference intrinsic inference is a formal answer to such kind of problem. See this example and references given therein. $\endgroup$ – Stéphane Laurent Jul 22 '14 at 8:26
  • $\begingroup$ What is a "vague" hypothesis by the way ? $H_0\colon\{\theta=\theta_0\}$ doesn't sound vague for me. $\endgroup$ – Stéphane Laurent Jul 22 '14 at 8:34
  • $\begingroup$ I have just taken a look at Sprenger's paper mentionned in your edit. The Bayesian reference criterion (BRC) is more or less what I called "reference intrinsic inference" in my previous comment. $\endgroup$ – Stéphane Laurent Jul 22 '14 at 8:37
1
$\begingroup$

The questions depends greatly on if you're sold on hypothesis tests per se, versus parameter estimation techniques. Bayesian hypothesis testing has received much of the same criticism as frequentist hypothesis testing. One alternative, which ameliorates the problems you're concerned about, is parameter estimation in the sense @John Kruschke explains here. Kruschke discusses the two approaches in detail on his blog.

The method works like this: you set up a region of possible effects which you consider insignificant - where insignificant is meant in the colloquial sense, not the statistical sense. For example, you could say that coefficients or subject means or a Cohen's d between -0.1 and +0.1 entail that the relationship is practically uninteresting and without implications. Then, you produce a Bayesian credible interval for your data. If the interval is wholly contained within your region, your Bayes-optimal decision would be to assume the irrelevance of the relationship. For this, it is not even relevant if 0 itself is in the interval.

It is actually rather hard to "prove the null" in this way because to reach a CI small enough to fit inside a non-trivial region requires a lot of data. Strong priors could stabilize your estimates towards 0 (leading to an interval that is more similar to your region), but unless they are overwhelmingly strong, they don't tend to strongly overestimate the support for such a null, since narrow intervals depend on rich data.

If you're too concerned, you could also try either very weak priors on the variance parameter, or uniform priors on the parameter itself. And as always, simulate some worst-case data and check the bias of your test.

| cite | improve this answer | |
$\endgroup$
0
$\begingroup$

If you have weak priors, then your posterior distributions are (almost) entirely determined by your sample data (as in the frequentist approach). If your sample data are not consistent with your null hypothesis, then the null hypothesis is not going to be supported.

I think what you read might have been about having strong (non-vague) priors that are not in the direction of the "null hypothesis". In the situation where your sample data are in fact consistent with the null hypothesis, then you are less likely to actually find support for the null if you use non-vague priors that are inconsistent with the null than if you use vague priors (and this is true even though both analyses would use the same sample data). So for "vague priors lend support to the null hypothesis" to be true, you'd need your sample data to be consistent with the null hypothesis. (Also, note that nothing prevents a non-vague prior to also be consistent with the null hypothesis, in which case it becomes even easier to find support for the null.)

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.