I've seen questions like this and this, but it hasn't quite answered my question. How intertwined are the Beta and Binomial distributions?

A quick sidenote: the Poisson distribution and the exponential distribution are two faces of the same coin. The Poisson distributions is a count process for events whereas the exponential distribution models time between those same events.

I know the Beta distribution is a convenient way to model the uncertainty in $p$ in a Binomial distribution because the Beta distribution is flexible (with the $\alpha$ and $\beta$ parameters) and exists between 0 and 1 (just like probabilities should be). But is there more to the story? Are the Beta distribution and Binomial distribution two faces of the same coin as well?