1
$\begingroup$

Decades ago improper objective priors - e.g. $\pi(\sigma) \propto \sigma^{-1}, \sigma > 0,$ for a scale parameter - were considered problematic because some authors thought they were leading to the so-called "marginalization paradox". It seems that this issue has been resolved by Jaynes in his book Probability Theory - The Logic of Science, Section 15.8. Therefore my question: Is there any strong argument about objective/non-informative improper prior ?

$\endgroup$
6
  • 1
    $\begingroup$ Does this answer your question? What is an "uninformative prior"? Can we ever have one with truly no information? $\endgroup$
    – Xi'an
    Apr 16, 2022 at 13:40
  • $\begingroup$ Thank you for your quick answer. No I don't think it answers my question, because I don't see any mention of an fundamental/unsolvable problem with improper priors. Unless that is the answer I shall understand : there is no problem with improper priors ? : ) $\endgroup$
    – Celi
    Apr 16, 2022 at 13:49
  • $\begingroup$ The msg of Jaynes in Sec. 15.8 is that marginalization paradox arises because a conditioning event that has probability 0 may carry different information depending on how the problem is parametrized, even if it doesn't look so. The paradox is resolved when the conditioning event with probability 0 is expressed as the limit of an event that has a positive mass of probability. $\endgroup$
    – Celi
    Apr 16, 2022 at 13:58
  • 1
    $\begingroup$ If you found this answer helpful, then please consider upvoting and/or accepting it. $\endgroup$ Apr 22, 2022 at 16:11
  • 1
    $\begingroup$ Without claiming to have deep understanding of the subject, it appears Jaynes not so much solved the marginalization paradox as he did apply a different notion of a limit (compared to the original 'discoverers' of the marginalization paradox; Dawid, Stone, and Zidek). $\endgroup$
    – Durden
    Jun 22, 2023 at 17:41

1 Answer 1

1
$\begingroup$

enter image description hereMarginalisation paradoxes are fascinating and I always mention them in my Bayesian class, because I think they illustrate the limitations of how much one can interpret an improper prior. There is a consequent literature on how to “solve” marginalisation paradoxes, following Jaynes’ comments on the foundational paper of David, Stone and Zidek (Journal of the Royal Statistical Society, 1974), but I do not think they need to be “solved” either by uncovering the group action on the problem (left Haar versus right Haar) or by using different proper prior sequences. For me, the core of the “paradox” is that writing an improper prior as $$ \pi(\theta,\zeta) = \pi_1(\theta) \pi_2(\zeta) $$ does not imply that $\pi_2$ is the marginal prior on $\zeta$ when $\pi_1$ is improper. The interpretation of $\pi_2$ as such is what leads to the “paradox” but there is no mathematical difficulty in the issue. Starting with the joint improper prior $\pi(\theta,\zeta)$ leads to an undefined posterior if we only consider the part of the observations that depends on $\zeta$ because $\theta$ does not integrate out. Defining improper priors as limits of proper priors—as Jaynes does—can also be attempted from a mathematical point of view, but (a) I do not think a global resolution is possible this way in that all Bayesian procedures for the improper prior cannot be constructed as limits from the corresponding Bayesian procedures for the proper prior sequence, think eg about testing, and (b) this is trying to give a probabilistic meaning to the improper priors and thus gets back to the over-interpretation danger mentioned above.

$\endgroup$
7
  • $\begingroup$ I think there is an error in point a. : the posterior does depend on x1, in addition to z, via an indicator function that ensures x1 is positive (the pdf of an exponential random variable being defined for positive values only) ... that is, the posterior must be written as g(z) * 1{x1 > 0} ... or perhaps 1{x1 >= 0}, depending on how the exponential distribution is defined $\endgroup$
    – Celi
    Apr 16, 2022 at 16:38
  • $\begingroup$ Why should the posterior be multiplied by this indicator as a function of the parameter? The indicators all equal one, given the observations. $\endgroup$
    – Xi'an
    Apr 16, 2022 at 16:42
  • $\begingroup$ Say you have calculated that posterior, which is supposed to depend on z only. Then I come to you with a vector z, and ask you what's the value of the posterior for that z, at e.g. xi = 1. Then you tell me a positive value. And then I tell you that in fact I calculated z with only negative values for x1, ..., xn. : ) So the postive value should have been a 0 ! And hence knowing z only seems not suffcient for evaluating that posterior. ... and hence in fact we need all the indicators, not just the one for x1 $\endgroup$
    – Celi
    Apr 16, 2022 at 16:58
  • 1
    $\begingroup$ Please see stats.meta.stackexchange.com/questions/6304/my-upvoting-policy, when you find a question sufficiently clear to write an answer, consider to upvote the question! $\endgroup$ Apr 21, 2022 at 15:04
  • 1
    $\begingroup$ If you found this answer helpful, then please consider upvoting and/or accepting it. $\endgroup$ Apr 22, 2022 at 16:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.