1
$\begingroup$

Book "Bayesian Statistics the Fun Way: Understanding Statistics and Probability with Star Wars, Lego, and Rubber Ducks", chapter 9 "Bayesian priors and working with probability distributions".

In the chapter, the author tried to demonstrate how to "use probability distributions to numerically describe our beliefs as a range of possible values rather than single values".

The author used the scene in Star Wars: the Empire Strikes Back where Han Solo, attempting to evade enemy fighters, files the Millennium Falcon into an asteroid field as an example.

The author suggested that the posterior (Han making through the asteroid field) is calculated from the C-3PO beliefs (likelihood) and our beliefs in Han's badassery (prior).

Here a summarized of steps the author proposed to calculate the posterior:

  1. Likelihood

"Sir, the possibility of successfully navigating an asteroid field is approximately 3,720 to 1!"

The author suggested to use beta distribution:

Because C-3PO provides the approximate odds of successfully navigating an asteroid field, we know that the data he has gives him only enough information to suggest a range of possible rates of success. To represent that range, we need to look at a distribution of beliefs regarding the probability of success, rather than a single value representing the probability.

Then the author provided the formula:

P(RateOfSuccess | Successes and Failures) = $Beta(\alpha, \beta)$

From what I understand, apply that to the problem mean:

Let RateOfSuccess_likelihood = The success rate of navigating an asteroid field is approximately 3,720 to 1

P(RateOfSuccess_likelihood | succeses and failures) = $Beta(\alpha_{likelihood}, \beta_{likelihood})$

The author then gave an assumption: $\alpha$ = 2, $\beta$ = 7400

  1. Prior

The author also suggested to use beta distribution to model the prior beliefs.

Given Han's badassery (plot armor), the author assumed: $\alpha$ = 20000, $\beta$ = 1

Therefore:

Let RateOfSuccess_prior = The success rate of navigating an asteroid field is approximately 2 to 20000

P(RateOfSuccess_prior | successes and failures) = $Beta(\alpha_{prior},\beta_{prior})$

  1. Posterior

The author tried to use the formula discussed in previous chapter to calculate the posterior distribution:

Posterior $\propto$ Likelihood $\times$ Prior

Remember, using this proportional form of Bayes' theorem means that our posterior distribution doesn't necessarily sum to 1. But we're lucky because there's an easy way to combine beta distributions that will give us a normalized posterior when all we have is the likelihood and the prior. Combining our two beta distributions - one representing C-3PO's data (the likelihood) and the other our prior belief in Han's ability to survive anything (our prior) - in this way is remarkably easy:

$ Beta(\alpha_{posterior},\beta_{posterior}) = Beta(\alpha_{likelihood}+\alpha_{prior},\beta_{likelihood}+\beta_{prior}) $

The author comes up with the formula

$Beta(\alpha_{posterior},\beta_{posterior}) = Beta(\alpha_{likelihood}+ \alpha_{prior}, \beta_{likelihood}+ \beta_{prior})$

without any explanation.

This is the part where I don't understand. How can the author comes up with the above formula?


Here my attempts to figure out:

  1. Substitutes beta distribution formula

Posterior $\propto$ Likelihood $\times$ Prior,

$\rightarrow Beta(\alpha_{posterior}, \beta_{posterior})$

$= Beta(\alpha_{likelihood}, \beta_{likelihood}) \times Beta(\alpha_{prior},\beta_{prior})$

$= x^{\alpha_{likelihood}- 1}(1 - x)^{\beta_{likelihood}- 1}x^{\alpha_{prior}- 1}(1 - x)^{\beta_{prior}- 1}$

$= x^{\alpha_{likelihood}+ \alpha_{prior}- 2}(1 - x)^{\beta_{likelihood}+ \beta_{prior}- 2}$

$= Beta(\alpha_{likelihood}+ \alpha_{prior}- 1, \beta_{likelihood}+ \beta_{prior}- 1)$

which is different with the formula the author come up with.

  1. After some further search from the Internet, I found some links to articles related to the problem. Here are two of them:

Conjugate prior

Help me understand Bayesian prior and posterior distributions

The result formula in these links are similar to that of the author. E.g:

Posterior distribution

Which is equivalent to $Beta(s + \alpha, f + \beta)$ which is similar to the formula the author came up with $Beta(\alpha_{likelihood}+ \alpha_{prior}, \beta_{likelihood}+ \beta_{prior})$

However, in these articles and the others I have found, the likelihood is binomial rather than beta distribution as suggested by the author. Why?

Is the author wrong or the examples in the book and the links I found are different cases? I would appreciate any explanation (i.e. How the author come up with the formula) and/or link to articles mentioned usages of Bayes' theorem and beta distribution likelihood.

(Most of my knowledge in statistics comes from self-education, thus I still have many gaps in understanding concepts that may seem trivial for other people here. So I would be very thankful if answers included less specific terms and more explanation).

Thank you for any help you can provide.

$\endgroup$

1 Answer 1

2
$\begingroup$

Indeed, beta is a conjugate prior for binomial distribution. On another hand, beta is not conjugate prior for beta distribution, so there is no simple, closed-form, solution for the posterior of such model.

So if $k$ is the number of successes and $n$ is the total number of trials (hence $n-k$ is the number of failures), then we can use binomial as a likelihood

$$ k|p \sim \mathsf{Binom}(n, p) $$

and beta as a prior

$$ p \sim \mathsf{Beta}(\alpha, \beta) $$

then the posterior would be

$$ p | k \sim \mathsf{Beta}(\alpha + k, \beta + n - k) $$

Maybe, what author means is that C3PO’s distribution is a prior and then the description jumps to describing posterior, while skipping the likelihood part?

$\endgroup$
4
  • $\begingroup$ Hi Tim, thank you for the answer. I found the article by the author that the chapter in the book is based on. Could you reconfirm whether it is the author wrong or I misunderstand something? (I'm concerned that because I misunderstand something, my question does not capture exactly what the author means). Thank you very much. $\endgroup$ Jul 3, 2020 at 2:42
  • $\begingroup$ @SweetPotato see my edit for possible interpretation $\endgroup$
    – Tim
    Jul 3, 2020 at 4:18
  • $\begingroup$ In the article, the author said: "In Bayesian terms, C3PO's estimate of the true rate of success given observed data is referred to as the Likelihood", "We have now established what C3PO believes (Likelihood) and modeled our own beliefs in Han (Prior Probability), but we need a way to combine these. " and "It also turns out that combining our two Beta distributions in this way, including the normalization, is remarkably easy." $\endgroup$ Jul 3, 2020 at 4:29
  • $\begingroup$ @SweetPotato so either it is an error or the description is just confusing. Otherwise, this would be incorrect, as stated in the answer. $\endgroup$
    – Tim
    Jul 3, 2020 at 4:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.