Assume I want to apply Bayes theorem with some state variable $x$ (scalar or vector, doesn't matter) and an observation vector $\mathbf{y}=[y_1,...,y_N]^T$::


Further assume that my likelihood is composite, i.e.:

$$p(\mathbf{y}|x)=\prod_{i=1}^N p(y_i|x)$$

where the $y_i$ are independent observations. Would it be possible to also decompose the model evidence $p(\mathbf{y})$ into its components as:


If not, why?

  • 1
    $\begingroup$ The denominator is incorrect because the $y_i$'s become dependent once $x$ is integrated out. $\endgroup$
    – Xi'an
    Sep 15, 2021 at 11:47

2 Answers 2


We can obtain some intuition from an example.

Let $p(y_i|x) = \textsf{N}(y_i|x,1)$, $p(x) = \textsf{N}(x|0,s^2)$, and let $N = 2$. Then \begin{equation} p(y) = \textsf{N}(y|0_2,\Sigma) \end{equation} where $y = (y_1,y_2)$, $0_2 = (0,0)$, and \begin{equation} \Sigma = \begin{pmatrix} s^2 + 1 & s^2 \\ s^2 & s^2 + 1 \end{pmatrix} . \end{equation} The point is that variation in $x$ induces dependence among the $y_i$.

In this example, we can eliminate the variation in $x$ by letting $s$ go to zero, thereby eliminating the dependence among the $y_i$: \begin{equation} \lim_{s\to 0}\ \Sigma = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} . \end{equation} In the limit, $p(y) = p(y_1)\,p(y_2)$. The limiting case amounts to putting a point-mass prior on $x$.


You can, yes. In the more general case, if we also expand the numerator in the same way, we'd get the following:

$$p(x|\mathbf{y}) = p(x) \frac{p(y_1|x)}{p(y_1)} \frac{p(y_2|x, y_1)}{p(y_2|y_1)} \ldots \frac{p(y_N|x, y_1, \ldots, y_{N-1})}{p(y_N | y_1, \ldots, y_{N-1})},$$

where the conditioning on previous $y$ in the numerator and the denominator vanishes when the observations are independent, which is the case you're looking at.

This decomposes the likelihood/marginal ratio into a ratio for each observation, conditional on previous ones. Sticking with the general case, if we then collapse these terms one by one...

$$\begin{align*} p(x|\mathbf{y}) &= p(x|y_1) \frac{p(y_2|x, y_1)}{p(y_2|y_1)} \ldots \frac{p(y_N|x, y_1, \ldots, y_{N-1})}{p(y_N | y_1, \ldots, y_{N-1})}\\ &= p(x|y_1, y_2) \frac{p(y_3|x, y_1, y_2)}{p(y_3|y_1, y_2)} \ldots \frac{p(y_N|x, y_1, \ldots, y_{N-1})}{p(y_N | y_1, \ldots, y_{N-1})}\ldots\\ &= p(x|y_1, y_2, \ldots, y_N), \end{align*}$$

we can see that this is equivalent to finding the posterior conditional on $y_1$, then using this as the prior for observing $y_2$, and so on. Hence, we can decompose the model evidence as if we'd only taken one observation before each model update.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.