2
$\begingroup$

When we toss a coin with an unknown Heads probability $p$, we can use Bayesian inference to estimate the unknown value $p$.

Say, we start with a Beta prior distribution with parameters $(a,b)$ and then update the prior as we observe Heads or Tails.

At the point we tossed $n$ times and have observed $n_H$ number of Heads, we say the probability of Head is $$\hat p_H=\frac{a+n_H}{a+b+n}=\frac{a}{a+b+n}+\frac{n_H}{a+b+n}.$$

Here, as we can see, if $n_H$ is increases by one, the estimate $\hat p_H$ increase by $1/(a+b+n)$.

I wonder if there is an intuitive explanation of why this increment of the estimate decreases as $n$ increases. Any idea?

$\endgroup$
3
$\begingroup$

Because the posterior (or, more precisely, the posterior mean that you state here) is formed by weighting over the contribution from the prior and the observations.

Hence, if $n$ increases, the contribution of any particular "head" to the total estimate decreases. When you move from $n=1$ to $n=2$ the first observation still is pretty important to form the posterior mean, while if $n=1,000$, any particular observation is relatively less influential.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.