# Apply Bayes rule sequentially

I have

$$\pi(a) =P(a|b) = \frac{P(b|a)P(a)}{P(b)}$$

I would like to now update given some new information, $C=c$

Is it possible to write:

$$P(a|b,c) = \frac{P(b,c|a)P(a)}{P(b,c)} \stackrel{?}{=} \frac{P(c|a)\pi(a)}{P(c)}$$

You can write:

$$P(a,b,c) = P(a \vert b,c)P(b,c) = P(a \vert b,c)P(c \vert b)P(b)$$

or, also valid:

$$P(a,b,c) = P(c \vert a,b)P(a,b) = P(c \vert a,b)P(b \vert a)P(a)$$

Putting together both expressions:

$$P(a \vert b,c) = \frac{P(c \vert a,b)P(b \vert a)P(a)}{P(c \vert b)P(b)} = \frac{P(c \vert a,b) \pi(a)}{P(c \vert b)}$$

And if this new observation $$c$$ does not depend on the previous observation $$b$$ (i.e. $$P(b,c) = P(b)P(c)$$), you can write:

$$P(a \vert b,c) = \frac{P(c \vert a) \pi(a)}{P(c)}$$

• Thank you, this answers the question. For something like a signal, however, doesn't the next observation usually depend on the previous one? – user0 Jun 6 '18 at 16:48
• @user2357111317, actually that will depend on the autocorrelation of that signal – Carlos Campos Jun 7 '18 at 9:37
• I especially appreciate your method of solving this by essentially deriving Bayes rule by showing how the joint equals the products in each case - it allowed me to solve a similar, more difficult problem. – user0 Jan 7 at 5:19

It might help to use some more specific notation because the Bayesian update will depend on what model you're using.

As an example, say you had a linear regression model where you regressed $$y_i$$ on $$\mathbf{x}_i$$ variables. The coefficients/parameters for this model could be called $$\theta$$. If you had a batch of $$n$$ data points, you might write your likelihood as $$p(\mathbf{y}_{1:n} \mid \mathbf{X}_{1:n}, \theta) = \prod_{i=1}^n p(y_i \mid \mathbf{x}_i, \theta).$$ Your model would be complete as soon as you chose some prior distribution $$\pi(\theta)$$.

Bayes' rule states $$\pi(\theta \mid \mathbf{y}_{1:n}, \mathbf{x}_{1:n}, \theta) \propto p(\mathbf{y}_{1:n} \mid \mathbf{X}_{1:n}, \theta)\pi(\theta).$$ If you got another row of data ($$y_{n+1}, \mathbf{x}_{n+1}$$), then you could update your posterior using $$\pi(\theta \mid \mathbf{y}_{1:n+1}, \mathbf{x}_{1:n+1}, \theta) = p(y_{n+1} \mid \mathbf{x}_{n+1}, \theta)\pi(\theta \mid \mathbf{y}_{1:n}, \mathbf{x}_{1:n}, \theta).$$ So the old posterior distribution takes the place of the prior distribution when you update sequentially.

As I said before, the Bayesian update will depend on what model you're using. A different model would have different conditional independence structure. However, many other models will resemble the last expression in that the old posterior is used as a prior, and it's being multiplied by some marginal "likelihood."