A question on Bayesian updating (Han Solo and the Bayesian priors)

Question

http://www.statslife.org.uk/the-statistics-dictionary/2148-han-solo-and-bayesian-priors

in the preceding article, there is a bit of Bayesian updating that does not make sense to me. We have a prior probability (of Han Solo escaping the asteroid field) and the C3PO estimate of a probability that someone survives the field.

they are somehow combined to yield a posterior probability (of Han surviving the field)

now ... the way I understood the process of calculating a posterior, you'd need a prior $P(A)$, observe an event $E$, and use two conditionals $P(E|A)$ and $P(E|\not A)$ to calculate the posterior (or, perhaps, the ratio between these two conditionals)

---

the question is: why is C3P0's estimate of the probability of a random person being used in this place? It seems you would need two conditionals, or a likelihood ratio, and C3P0's estimate is neither.

Answer 1

It is wrong to say that the Posterior = Prior*Likelihood as there is no guarantee that multiplying two distributions will result in a valid distribution (i.e, integrates to 1). However, it is perfectly legitimate to write Posterior $\propto$ Likelihood * Prior as there always some normalizing constant that will make the posterior a valid distribution.

You may be used to seeing the posterior as the following,

$$p(\theta|x) = \frac{p(x|\theta)p(\theta)}{p(x)}.$$

the denominator, $p(x)$, is the "normalizing constant". It is easy to see that this normalizing constant makes the posterior integrate to one if we notice $p(x)=\int^\infty_{-\infty} p(x|\theta)p(\theta)dx$. That is, $$\int^\infty_{-\infty}\frac{p(x|\theta)p(\theta)}{p(x)} dx = \frac{1}{p(x)}\int^\infty_{-\infty} p(x|\theta)p(\theta) dx =1$$

Since $p(x)$ is just some constant that make the posterior a valid distribution, it is not interesting and that's why one sees Posterior $\propto$ Likelihood * Prior or sometimes, incorrectly, Posterior = Likelihood*Prior.

Answer 2

I think in the article you are linking the authors avoid to consider the denominator and simply multiplicate the likelihood (C3PO) estimate by the prior (our guess that Han Solo will succeed).

Indeed, just after

Posterior = Likelihood $\times$ Prior

you read

The only thing not explicitly stated in this formula is that we usually want to normalize everything so it sums up to 1. It also turns out that combining our two beta distributions in this way, including the normalization, is remarkably easy.

Answer 3

I think your question is \begin{align} &H:= \text{Han survives} \\ &D:= \text{random pilots die} \\ \\ &P(H|D) \propto P(D|H) P(H) \\ &P(D|H) = ?! \end{align}

The article is guilty of some hand waving because in order for us to rationally update our belief in Han's chances based on other pilots' experience we have to establish a link between their performance and his. The update performed in the article was done as if the other pilots were an acceptable proxy for Han. However, the article wants to say at the same time that Han is somehow more awesome than the other pilots, which would break the applicability of their data.

One key thing to keep in mind is that this example is not about the probability of Han surviving per se but of our belief that he will survive. As first time viewers approaching the asteroid field, we don't know what the probability of survival for an attempted crossing by Han would be. We believe that chances are more likely high for Han than not, but we are uncertain. So, not only is the outcome uncertain for a given chance, we aren't even certain what the chances are.

Han is a pilot of extraordinary skill, so our belief in his chances of survival of a danger is a distribution $h$, heavily weighted toward 1. In this example, the model used is

\begin{equation} h(p) \sim Beta(20000,1) \propto p^{20000-1}(1-p)^{1-1} \end{equation}

so that our mean prior survival odds for Han are $E[h(p)] = 20000:1$.

Now, if C3PO provides data, $D$, that 2 pilots have survived and 7440 have perished, then for a given chance of survival $p$, the likelihood of that data is

\begin{equation} L(D|p) \propto p^2 (1-p)^{7440} \end{equation}

Even though he's really awesome, Han's chances of success do depend on the level of danger, which we now have reasons to believe are higher than before C3PO provided information. However, the following update only works to the extent the additional data is applicable as proxies for Han, that is to the extent we can treat their experiences as belonging to Han's sampling distribution.

\begin{align} h(p|D) &\propto L(D|p) h(p) \\ &\propto [p^2 (1-p)^{7440}][p^{20000-1}(1-p)^{1-1}] \\ &= p^{(20000+2)-1}(1-p)^{(7440+1)-1} \end{align}

Note that what's changed is our belief in Han's chances, not his actual chances per se. Though, if Han's beliefs were constructed in the same way as ours, C3PO may have just eroded his confidence. To the extent confidence is good in dangerous situations and our heroes are always overconfident, never let robots report the odds.