2
$\begingroup$

Let $\{f(\cdot|\theta): \theta \in \Theta \}$ be a family of pdfs and let $\pi: \Theta \to \mathbb{R}$ be a prior. According to Bayes' theorem (as stated in, e.g., Casella and Berger), the posterior distribution $\pi(\cdot|x)$ is given by $$ \pi(\theta|x) = \frac{f(x|\theta) \pi(\theta)}{\int_{\Theta} f(x|\theta) \pi(\theta)\,d\theta}. $$

My questions:

  1. How do we define the posterior distribution for values of $x$ such that $\int_{\Theta} f(x|\theta) \pi(\theta)\,d\theta = 0$ ? Or do we just leave it undefined?

  2. If my reasoning is correct, $\int_{\Theta} f(x|\theta) \pi(\theta)\,d\theta > 0$ provided that the set $E_x := \{\theta \in \Theta: f(x|\theta)\, \pi(\theta) > 0 \}$ has positive measure. Is there anything else that we can say about the set of $x$ for which $\int_{\Theta} f(x|\theta) \pi(\theta)\,d\theta > 0$ ?

  3. If one is working with a model such that $\int_{\Theta} f(x|\theta) \pi(\theta)\,d\theta = 0$ for certain values of $x$ in the sample space, does this indicate that there is a problem with the model (either in our choice of pdf $f(\cdot|\theta)$ or our choice of prior $\pi$) ?

$\endgroup$
1
  • 1
    $\begingroup$ This signals a problem with either the sampling model or the prior. $\endgroup$
    – Xi'an
    Apr 9, 2023 at 20:12

1 Answer 1

1
$\begingroup$

If$$\int_{\Theta} f(x|\theta) \pi(\theta)\,\text d\theta = 0$$the Bayesian model is incompatible with the data $x$. For instance, if $$X\sim\mathcal U(0,\theta)\qquad\theta\sim\mathcal U(0,1)$$ and the realisation $x$ of $X$ is $x=2$, this realisation is incompatible with the model. The model must be modified.

$\endgroup$
3
  • $\begingroup$ Thanks for your answer, that makes sense. So suppose we haven't yet generated a sample, but we know that for at least one sample value $x$ we have $\int_{\Theta} f(x|\theta) \pi(\theta) \,d\theta = 0$. At this stage, would it be advised to simply pick a different model? Or should we only be concerned after we obtain a sample value for which the integral is zero? $\endgroup$
    – Leonidas
    Apr 10, 2023 at 12:43
  • 1
    $\begingroup$ Very good sub-question: it is more coherent to throw away the model before observing $X=x$, as the Bayesian posterior need be defined for all possible realisations. Otherwise, the prior could end up depending on the realisation if it is the one to blame in the pair (sampling,prior). $\endgroup$
    – Xi'an
    Apr 10, 2023 at 16:03
  • $\begingroup$ Makes sense, thank you! $\endgroup$
    – Leonidas
    Apr 10, 2023 at 16:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.