I have

$$\pi(a) =P(a|b) = \frac{P(b|a)P(a)}{P(b)}$$

I would like to now update given some new information, $C=c$

Is it possible to write:

$$P(a|b,c) = \frac{P(b,c|a)P(a)}{P(b,c)} \stackrel{?}{=} \frac{P(c|a)\pi(a)}{P(c)} $$


You can write:

$$P(a,b,c) = P(a \vert b,c)P(b,c) = P(a \vert b,c)P(c \vert b)P(b)$$

or, also valid:

$$P(a,b,c) = P(c \vert a,b)P(a,b) = P(c \vert a,b)P(b \vert a)P(a)$$

Putting together both expressions:

$$P(a \vert b,c) = \frac{P(c \vert a,b)P(b \vert a)P(a)}{P(c \vert b)P(b)} = \frac{P(c \vert a,b) \pi(a)}{P(c \vert b)} $$

And if this new observation $c$ does not depend on the previous observation $b$ (i.e. $P(b,c) = P(b)P(c)$), you can write:

$$P(a \vert b,c) = \frac{P(c \vert a) \pi(a)}{P(c)} $$

  • 1
    $\begingroup$ Thank you, this answers the question. For something like a signal, however, doesn't the next observation usually depend on the previous one? $\endgroup$ – user0 Jun 6 '18 at 16:48
  • 1
    $\begingroup$ @user2357111317, actually that will depend on the autocorrelation of that signal $\endgroup$ – Carlos Campos Jun 7 '18 at 9:37
  • 2
    $\begingroup$ I especially appreciate your method of solving this by essentially deriving Bayes rule by showing how the joint equals the products in each case - it allowed me to solve a similar, more difficult problem. $\endgroup$ – user0 Jan 7 '19 at 5:19

It might help to use some more specific notation because the Bayesian update will depend on what model you're using.

As an example, say you had a linear regression model where you regressed $y_i$ on $\mathbf{x}_i$ variables. The coefficients/parameters for this model could be called $\theta$. If you had a batch of $n$ data points, you might write your likelihood as $$ p(\mathbf{y}_{1:n} \mid \mathbf{X}_{1:n}, \theta) = \prod_{i=1}^n p(y_i \mid \mathbf{x}_i, \theta). $$ Your model would be complete as soon as you chose some prior distribution $\pi(\theta)$.

Bayes' rule states $$ \pi(\theta \mid \mathbf{y}_{1:n}, \mathbf{x}_{1:n}, \theta) \propto p(\mathbf{y}_{1:n} \mid \mathbf{X}_{1:n}, \theta)\pi(\theta). $$ If you got another row of data ($y_{n+1}, \mathbf{x}_{n+1}$), then you could update your posterior using $$ \pi(\theta \mid \mathbf{y}_{1:n+1}, \mathbf{x}_{1:n+1}, \theta) = p(y_{n+1} \mid \mathbf{x}_{n+1}, \theta)\pi(\theta \mid \mathbf{y}_{1:n}, \mathbf{x}_{1:n}, \theta). $$ So the old posterior distribution takes the place of the prior distribution when you update sequentially.

As I said before, the Bayesian update will depend on what model you're using. A different model would have different conditional independence structure. However, many other models will resemble the last expression in that the old posterior is used as a prior, and it's being multiplied by some marginal "likelihood."


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.