Skip to main content
edited tags
Link
Christoph Hanck
  • 34.8k
  • 3
  • 78
  • 136
Source Link
Andeanlll
  • 423
  • 4
  • 13

Why marginal impact of an observation to the posterior decreases in Bayesian inference?

When we toss a coin with an unknown Heads probability $p$, we can use Bayesian inference to estimate the unknown value $p$.

Say, we start with a Beta prior distribution with parameters $(a,b)$ and then update the prior as we observe Heads or Tails.

At the point we tossed $n$ times and have observed $n_H$ number of Heads, we say the probability of Head is $$\hat p_H=\frac{a+n_H}{a+b+n}=\frac{a}{a+b+n}+\frac{n_H}{a+b+n}.$$

Here, as we can see, if $n_H$ is increases by one, the estimate $\hat p_H$ increase by $1/(a+b+n)$.

I wonder if there is an intuitive explanation of why this increment of the estimate decreases as $n$ increases. Any idea?