Consider a binomial model with beta prior $\theta\sim Beta(\alpha,\beta)$ for which we have observed $k$ successes in $n$ attempts. The MLE is $\hat\theta=k/n$, and the posterior mean is
$$
E(\theta|y)=\frac{\alpha+k}{\alpha+\beta+n}
$$
We see that $E(\theta|x)=\hat\theta$ if $\alpha=\beta=0$. This is known as the Haldane prior $\pi(\theta)\propto \theta^{-1}(1-\theta)^{-1}$, which is improper in view of $\int\theta^{-1}(1-\theta)^{-1}d\theta=\ln(\theta)-\ln(1-\theta)+C$ (and strictly speaking not a Beta prior, as it does not have positive parameters). Plotting it (in fact, an approximation with $\alpha,\beta\approx0$) shows it to have an extreme shape of a bathtub, putting all prior density on either 0 or 1:

theta <- seq(0,1,by=.000001)
plot(theta, dbeta(theta,.001,.001), type="l", lwd=2,
col="lightblue", ylim=c(0,10), ylab="Haldane approximation")
That leads to a behavior in which, when $k=n$ ($k=0$), we only expect successes (failures) in the future, as the posterior then "switches" to the value of $\theta$ that receives support by the data.