Bayesian priors and probability distributions

Question

Book "Bayesian Statistics the Fun Way: Understanding Statistics and Probability with Star Wars, Lego, and Rubber Ducks", chapter 9 "Bayesian priors and working with probability distributions".

In the chapter, the author tried to demonstrate how to "use probability distributions to numerically describe our beliefs as a range of possible values rather than single values".

The author used the scene in Star Wars: the Empire Strikes Back where Han Solo, attempting to evade enemy fighters, files the Millennium Falcon into an asteroid field as an example.

The author suggested that the posterior (Han making through the asteroid field) is calculated from the C-3PO beliefs (likelihood) and our beliefs in Han's badassery (prior).

Here a summarized of steps the author proposed to calculate the posterior:

1. Likelihood

"Sir, the possibility of successfully navigating an asteroid field is approximately 3,720 to 1!"

The author suggested to use beta distribution:

Because C-3PO provides the approximate odds of successfully navigating an asteroid field, we know that the data he has gives him only enough information to suggest a range of possible rates of success. To represent that range, we need to look at a distribution of beliefs regarding the probability of success, rather than a single value representing the probability.

Then the author provided the formula:

P(RateOfSuccess | Successes and Failures) = $Beta(\alpha, \beta)$

From what I understand, apply that to the problem mean:

Let RateOfSuccess_likelihood = The success rate of navigating an asteroid field is approximately 3,720 to 1

P(RateOfSuccess_likelihood | succeses and failures) = $Beta(\alpha_{likelihood}, \beta_{likelihood})$

The author then gave an assumption: $\alpha$ = 2, $\beta$ = 7400

2. Prior

The author also suggested to use beta distribution to model the prior beliefs.

Given Han's badassery (plot armor), the author assumed: $\alpha$ = 20000, $\beta$ = 1

Therefore:

Let RateOfSuccess_prior = The success rate of navigating an asteroid field is approximately 2 to 20000

P(RateOfSuccess_prior | successes and failures) = $Beta(\alpha_{prior},\beta_{prior})$

3. Posterior

The author tried to use the formula discussed in previous chapter to calculate the posterior distribution:

Posterior $\propto$ Likelihood $\times$ Prior

Remember, using this proportional form of Bayes' theorem means that our posterior distribution doesn't necessarily sum to 1. But we're lucky because there's an easy way to combine beta distributions that will give us a normalized posterior when all we have is the likelihood and the prior. Combining our two beta distributions - one representing C-3PO's data (the likelihood) and the other our prior belief in Han's ability to survive anything (our prior) - in this way is remarkably easy:

$ Beta(\alpha_{posterior},\beta_{posterior}) = Beta(\alpha_{likelihood}+\alpha_{prior},\beta_{likelihood}+\beta_{prior}) $

The author comes up with the formula

$Beta(\alpha_{posterior},\beta_{posterior}) = Beta(\alpha_{likelihood}+ \alpha_{prior}, \beta_{likelihood}+ \beta_{prior})$

without any explanation.

This is the part where I don't understand. How can the author comes up with the above formula?

---

Here my attempts to figure out:

1. Substitutes beta distribution formula

Posterior $\propto$ Likelihood $\times$ Prior,

$\rightarrow Beta(\alpha_{posterior}, \beta_{posterior})$

$= Beta(\alpha_{likelihood}, \beta_{likelihood}) \times Beta(\alpha_{prior},\beta_{prior})$

$= x^{\alpha_{likelihood}- 1}(1 - x)^{\beta_{likelihood}- 1}x^{\alpha_{prior}- 1}(1 - x)^{\beta_{prior}- 1}$

$= x^{\alpha_{likelihood}+ \alpha_{prior}- 2}(1 - x)^{\beta_{likelihood}+ \beta_{prior}- 2}$

$= Beta(\alpha_{likelihood}+ \alpha_{prior}- 1, \beta_{likelihood}+ \beta_{prior}- 1)$

which is different with the formula the author come up with.

2. After some further search from the Internet, I found some links to articles related to the problem. Here are two of them:

Conjugate prior

Help me understand Bayesian prior and posterior distributions

The result formula in these links are similar to that of the author. E.g:

Which is equivalent to $Beta(s + \alpha, f + \beta)$ which is similar to the formula the author came up with $Beta(\alpha_{likelihood}+ \alpha_{prior}, \beta_{likelihood}+ \beta_{prior})$

However, in these articles and the others I have found, the likelihood is binomial rather than beta distribution as suggested by the author. Why?

Is the author wrong or the examples in the book and the links I found are different cases? I would appreciate any explanation (i.e. How the author come up with the formula) and/or link to articles mentioned usages of Bayes' theorem and beta distribution likelihood.

(Most of my knowledge in statistics comes from self-education, thus I still have many gaps in understanding concepts that may seem trivial for other people here. So I would be very thankful if answers included less specific terms and more explanation).

Thank you for any help you can provide.

Answer 1

Indeed, beta is a conjugate prior for binomial distribution. On another hand, beta is not conjugate prior for beta distribution, so there is no simple, closed-form, solution for the posterior of such model.

So if $k$ is the number of successes and $n$ is the total number of trials (hence $n-k$ is the number of failures), then we can use binomial as a likelihood

$$ k|p \sim \mathsf{Binom}(n, p) $$

and beta as a prior

$$ p \sim \mathsf{Beta}(\alpha, \beta) $$

then the posterior would be

$$ p | k \sim \mathsf{Beta}(\alpha + k, \beta + n - k) $$

Maybe, what author means is that C3PO’s distribution is a prior and then the description jumps to describing posterior, while skipping the likelihood part?