# Bayesian and frequency tail estimation

The tail probability can be estimated by two methods:

1. In Bayesian method: $$P_B(X>a)=\int^{\infty}_{-\infty}\pi(\theta|x)[1-F(a|\theta)]d\theta$$
2. In Plug-in frequency method: $$P_F(X>a)=1-F(a|\hat{\theta})$$ where $$\hat{\theta}$$ is the MLE of $$\theta$$.

The numerical results show that it's always $$P_B \geq P_F$$ no matter what the distribution is.

Any ideas or any resources related to this topic to explain why is that?

First note that $$P_B(X > a)$$ is the value of the survival function of the predictive distribution, which is a mixture of distributions taken in the parametric family considered, the mixture weight being the posterior density.
I do not think that the result is always true, that is: for any distribution, for any sample, and for any $$a$$. However, the predictive distribution has tail which is thicker than any parametric distribution in the posterior which means that the inequality holds for large $$a$$. Under quite general conditions, the ML estimate $$\hat{\theta}$$ is in the support of the posterior, and the claimed relation should hold for large $$a$$. The reason is that a mixture of continuous distributions has a tail which is heavier than that of its components. I wrote some details in this answer to an apparently quite different question.
To get a counterexample we can choose a prior such that the ML estimate $$\hat{\theta}$$ does not always belong to it. For example consider an exponential distribution with mean $$\theta$$ assumed to lie in $$(0, \, 1)$$. If it happens that $$\hat{\theta} > 1$$ then the inequality will not hold.