So I have heard that if the prior distribution is in the subexponential class, applying Bayes rule does not change the belief. I have been trying to find an example of this but I am unable to do so. I was wondering if somebody around here has an example. This seems similar to the martingale property but I can't quite make the link...

  • 1
    $\begingroup$ When there's no data for one. $\endgroup$ – AdamO Oct 17 '18 at 13:31
  • 2
    $\begingroup$ I haven't heard of "subexponential distributions" 'til now, and the Wiki article on them is pretty dreadful. However, if I take this to mean any very long tailed distribution (say a Cauchy with median $\theta$?) , then the statement is simply not correct. Updating a very flat prior with data will lead to a posterior that's only a little different from the prior as compared to a prior in the exponential class. $\endgroup$ – AdamO Oct 17 '18 at 13:38
  • $\begingroup$ Apparently, fat tails are contained in subexponential, is there any way the statement can be true for a Pareto distribution? $\endgroup$ – Dio Oct 17 '18 at 13:45
  • $\begingroup$ Just use Gibbs Sampling; convince yourself the posterior always changes a little bit. $\endgroup$ – AdamO Oct 17 '18 at 15:00
  • $\begingroup$ The only issue is support. "With prior probability 0 that the moon is made of cheese, then astronauts carrying back armfuls of gruyere could not convince." However, even if the unrestricted MLE lies beyond the support, the posterior will still shift toward that value. $\endgroup$ – AdamO Oct 17 '18 at 15:02

For a given realisation, $x_0$, of $X\sim f(x|\theta)$, it is possible that $f(x_0|\theta)$ is a constant function of $\theta$ in which case the posterior is equal to the prior for this observation. However, if the posterior is almost surely equal to the prior, in the sense of almost every realisation of $X$, this implies that the density of $X$ does not depend on $\theta$.

I would welcome the reference to the subexponential distributions as there is no mathematical reason for a heavy or very heavy tail distribution to fail to update the prior. For instance, here is the comparison of prior (dashes) and posterior (full) when the prior is a Laplace (double-exponential) distribution and the likelihood is a Cauchy with observation 3.14:

enter image description here

The prior has clearly moved when becoming the posterior and the tail behaviour changed. A likelihood flatter than the Cauchy like a $t$ distribution with $10^{-1}$ or $10^{-2}$ degrees of freedom would exhibit the same pattern, albeit keeping the same mode for prior and posterior (as mentioned in my book).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.