Bayesian and frequency tail estimation The tail probability can be estimated by two methods:

*

*In Bayesian method:
$$P_B(X>a)=\int^{\infty}_{-\infty}\pi(\theta|x)[1-F(a|\theta)]d\theta$$

*In Plug-in frequency method:
$$P_F(X>a)=1-F(a|\hat{\theta})$$
where $\hat{\theta}$ is the MLE of $\theta$.

The numerical results show that it's always $$P_B \geq P_F$$
no matter what the distribution is.
Any ideas or any resources related to this topic to explain why is that?
 A: First note that $P_B(X > a)$ is the value of the survival function of
the predictive distribution, which is a mixture of distributions
taken in the parametric family considered, the mixture weight being the
posterior density.
I do not think that the result is always true, that is: for any distribution,
for any sample, and for any $a$. However, the predictive
distribution has tail which is thicker than any parametric
distribution in the posterior which means that the inequality holds
for large $a$. Under quite general conditions, the ML estimate $\hat{\theta}$
is in the support of the posterior, and the claimed
relation should hold for large $a$. The reason is
that a mixture of continuous distributions has a tail which is
heavier than that of its components. I wrote some details in this
answer to an
apparently quite different question.
To get a counterexample we can choose a prior such that the ML
estimate $\hat{\theta}$ does not always belong to it. For example
consider an exponential distribution with mean $\theta$ assumed to lie
in $(0, \, 1)$. If it happens that $\hat{\theta} > 1$ then
the inequality will not hold.
