What is the name of (and alternatives to) this Bayesian point-estimate?

Question

Assume that we have a Markovian environment that generates at every time step an event $A$ with probability $p^*$ and an event $B$ otherwise. Now suppose you are a Bayesian agent that wants to learn $p^*$ from the observations you've made so far. If we let our hypothesis class be $\{H_x\}_{0 \leq x \leq 1}$ and our initial prior be the uniform distribution ($f_0(x) = 1$), then according to Bayes rule we will update at each time step as:

$$ f_{t+1} = \begin{cases} \frac{x f_t(x)}{\int_0^1 x f_t(x) dx} & \text{if we saw A} \\ \frac{(1 - x) f_t(x)}{\int_0^1 (1 - x) f_t(x) dx} & \text{if we saw B} \end{cases} $$

The best estimate (at time $t$) $p_t$ for $p^*$ will then be the mean of $f_t$. I can show that tracking $p_t$ reduces to counting, in the sense that if by time $t$ you saw $a$ many $A$ events and $b = t - a$ many $B$ events then

$$p_t = \frac{a + 1}{(a + 1) + (b + 1)}$$

This seems like a basic result I would have seen in an introductory statistics course (which I unfortunately do not have). What is this called? If an agent was using the above procedure to estimate $p^*$ what kind of inference, regression, or estimation would they be doing?

Further, to make the answer not just a one-liner. What are some alternatives? In particular, what kind of assumptions would I need to make to have $p_t = \frac{a}{a + b}$? I.e. how can I eliminate the uniformity bias caused by my choice of prior?

Answer 1

The best estimate? Best is subjective in Bayesian inference. It can be shown that using the mean of $f_t$, as you have, is the best estimate in the expected squared error sense, i.e. it minimizes $E_{f_t}[(p_t - p^*)^2]$. The function $L(p_t, p^*) = (p_t - p^*)^2$ is the squared error loss function. But others can be considered:

• $L(p_t, p^*) = | p_t - p^* |$, and here $E_{f_t}[ L(p_t, p^*) ]$ is minimized by not the mean of the posterior, but the median of the posterior.

In general, the solution to the problem:

$$ \arg \min_{p_t} E_{f_t}[ L(p_t, p^*) ] $$ is called the Bayes action.

You can choose a Loss that accurate reflects your desires: maybe you want to bias slightly towards 0,1 end points, so you could choose the loss $$ L( p_t, p^*) = \frac{ (p_t - p^*)^2 }{ p_t(1-p_t) }$$

(Perhaps this fits into your rationality (or irrationality) model better as humans like to bias towards certainty as opposed to uncertainity). The Bayes action for this loss function would require more effort to calculate though.

Here is a good manuscript that explains Bayes actions and loss functions quite well.

The estimate $\frac{a}{a+b}$

The estimate you proposed actually has a very simple solution. That solution is the maximum likelihood estimator of $p^*$ which is equivilant to choosing a uniform prior (like you have) and choosing the maximum a-posterior estimate (the MAP estimate), defined as the arg max of the posterior. (see page 2 in the paper linked).

Answer 2

The Jeffreys' minimum informative prior for this case is a Beta(1/2,1/2) (it has a U shape in [0,1], not flat).

The posterior distribution for p* is a Beta(1/2+a,1/2+b)

You can search in the wikipedia for:

• Prior_probability
• Beta_distribution
• Jeffreys_prior