(Sorry if this is answered elsewhere, but I can't seem to find.)

Something about Bayesian updates that has always confused me - does the "current" posterior n-size matter? And if so, why not?

For instance, according to this (page 3), we have

$$ \tilde\pi_{n+1}(\theta) \propto \pi_{n}(\theta)f(x_{n+1}|\boldsymbol{x_n},\theta) $$

Seems to me, $n$ could be 10 or 10,000, and the "impact" of $x_{n+1}$ on $\pi$ would be the same. Shouldn't we give the "existing posterior" greater weight if there are more "existing" observations?

I'm sure I'm missing something, but can't seem to put my finger on it.

  • $\begingroup$ What do you mean by current" posterior n-size? Care to clarify? $\endgroup$ – kjetil b halvorsen Dec 11 '20 at 15:40
  • 1
    $\begingroup$ If $n$ is larger, $\pi_n(\theta)$ is likely to be a narrower distribution, which may correspond to your idea of "greater weight" $\endgroup$ – Henry Dec 11 '20 at 15:43
  • $\begingroup$ I mean you have seen n observations ("current") and want to update based on the n+1 observation. Where in the math does it account for the magnitude of n? $\endgroup$ – dashnick Dec 11 '20 at 15:43
  • $\begingroup$ @Henry is that true? Why does size of n have to do with the shape of the distribution? $\endgroup$ – dashnick Dec 11 '20 at 15:44
  • $\begingroup$ @dashnick Try an example yourself, for example a Bernoulli (or binomial) likelihood and a Beta prior and posterior distribution $\endgroup$ – Henry Dec 11 '20 at 15:50

The point of Bayesian sequential updating is that no mater if you update prior with likelihood over all of your data at once, or if you do this sequentially, the result would be the same. This wouldn't hold if different observations had different weights. If for some reason you need to assign different weights to different samples (e.g. survey weights), that this would work the same in sequential and all-at-once updating. However if you don't have good reason for doing so, usually you do not want different samples to affect the result to different degree.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.