Coin flipping: Relationship between Bayesian and Frequentist's point estimates I have a (biased) coin that has an unknown Head probability $p\in(0,1)$. To point estimate $p$, say that I'm going to use two approaches.
Approach 1. I can use the Bayesian inference technique. Starting from a Beta prior $p^0\sim Beta(a^0_H,a^0_T)$, I calculate the posterior from the observations. When I observed $n_H$ Heads and $n_T$ Tails in round $n_H+n_T$, The posterior will follow $Beta(a^0_H+n_H,a^0_T+n_T)$. As a point estimate for $p$, I can take the posterior mean, which is $\hat p_B=\frac{a^0_H+n_H}{a^0_H+n_H+a^0_T+n_T}$
Approach 2. I can use MLE. The MLE in this case is given by $\hat p_F=\frac{n_H}{n_H+n_T}$.
My question is what is the relationship between $\hat p_B$ and $\hat p_F$? Especially,

*

*It looks like when I have enough observations, the two estimates coincide. In this case, can I say $\hat p_B$ and $\hat p_T$ are asymptotically equal? or is there any other terminology that I can formally describe the asymptotic relationship between the two?

*When I only have a handful of observations, what can I say about the relationship? Can I say the two differ only by constant terms? or, again, is there any other formal description of the two under the small sample situation?

 A: The posterior point estimate is a weighted combination of
the prior point estimate,
$\frac{a^0_H}{a^0_H+a^0_T}$,
and the maximum likelihood estimate
$\frac{n_H}{n_H+n_T}$.
The weights are simply
$\omega_{\text{Prior}} = \frac{n_{\text{Prior}}}{n_{\text{Prior}} + n_{\text{Lik}}}$
and
$\omega_{\text{Lik}} = \frac{n_{\text{Lik}}}{n_{\text{Prior}} + n_{\text{Lik}}}$,
where
$n_{\text{Prior}} = a^0_H+a^0_T$
and
$n_{\text{Lik}} = n_H+n_T$.
You can confirm this by substitution:
$$
\begin{align}
\text{Posterior Mean}
 &= \frac{a^0_H}{a^0_H+a^0_T} \omega_{\text{Prior}} +
    \frac{n_H}{n_H+n_T} \omega_{\text{Lik}}\\
 &= \frac{a^0_H}{a^0_H+a^0_T} \times \frac{a^0_H + a^0_T}{a^0_H+a^0_T+n_H+n_T} +
    \frac{n_H}{n_H+n_T} \times \frac{n_H+n_T}{a^0_H+a^0_T+n_H+n_T}\\
 &= \frac{a^0_H}{a^0_H+a^0_T+n_H+n_T} +
    \frac{n_H}{a^0_H+a^0_T+n_H+n_T}\\
&= \frac{a^0_H+n_H}{a^0_H+n_H+a^0_T+n_T} 
\end{align}
$$


*

*With lots of observations, $\omega_{\text{Lik}} \gg \omega_{\text{Prior}}$,
and so the posterior mean is almost the same as the maximum likelihood estimate.

*With few observations, $\omega_{\text{Lik}} \ll \omega_{\text{Prior}}$,
and so the posterior mean is almost the same as the prior mean.

