# Particle Filtering: Derivation that mean of weights is the marginal likelihood

I see everywhere the following (for the Bootstrap Filter) $$p(y_t \mid y_{1:t-1}) \approx \frac{1}{N} \sum_{i=1}^N W(x_{0:t}^i)$$ where $$W(x_{0:t}^i)$$ are the normalized weights defined as

$$W(x_{0:t}^i) = \frac{w(x_{0:t}^i)}\sum_{j=1}^N w(x_{0:t}^j)}$$

where the unnormalized weights $$w(x^i_{0:t})$$ are $$w(x^i_{0:t}) = \frac{p(x_{0:t}^i \mid y_{1:t})}{q(x_{0:t}^i \mid y_{1:t})}$$

Everyone says this is straightforward to derive, but I really can't. Here's my attempt:

## My Attempt at solving the problem

\begin{align} p(y_t \mid y_{1:t-1}) &= \int p(y_t, x_{0:t} \mid y_{1:t-1}) dx_{0:t} \\ &= \int p(y_t \mid x_{0:t}) p(x_{0:t} \mid y_{1:t-1}) d x_{0:t} \\ &= \int p(y_t \mid x_t) p(x_t \mid x_{0:t-1} )p(x_{0:t-1} \mid y_{1:t-1}) d x_{0:t} && \text{y_t cond. indep.} \\ &= \int p(y_t \mid x_t)p(x_t \mid x_{t-1})p(x_{0:t-1} \mid y_{1:t-1}) dx_{0:t} && \text{x_t markovian} \\ &\approx \int p(y_t \mid x_t) p(x_t \mid x_{t-1}) \widehat{p}_N(x_{0:t-1}\mid y_{1:t-1}) d x_{0:t} && \text{empirical distr.} \\ &= \int p(y_t\mid x_t)p(x_t\mid x_{t-1}) \sum_{i=1}^N W(x_{0:t-1}^i)\delta_{x_{0:t-1}^i}(x_{0:t-1}) dx_{0:t} \\ &= \sum_{i=1}^N \int p(y_t\mid x_t)p(x_t\mid x_{t-1})W(x_{0:t-1}^i)\delta_{x_{0:t-1}^i}(x_{0:t-1}) dx_{0:t} \\ &= \sum_{i=1}^N \int p(y_t, x_t \mid x_{0:t-1}) W(x_{0:t-1}^i)\delta_{x_{0:t-1}^i}(x_{0:t-1}) dx_{0:t-1} dx_t \\ &= \sum_{i=1}^N \int p(y_t , x_t\mid x_{0:t-1}^i) W(x_{0:t-1}^i) dx_t \\ &= \sum_{i=1}^N p(y_t \mid x_{0:t-1}^i)W(x_{0:t-1}^i) \end{align}

where I have used the empirical approximation $$\widehat{p_N}(x_{0:t}) = \sum_{i=1}^N W(x_{0:t}^i)\delta_{x_{0:t}^i}(x_{0:t})$$

• You say you see this "everywhere", but can you provide a citation? From what I have seen, the bootstrap particle filter refers to an algorithm where you simulate directly from the prior (so mention of $q$ is a little out of place). In this case, the derivation simplifies significantly. For one references see, Algorithm 10.4 in Chopin & Papaspiliopoulos, 2020, Ch.10. Sep 4 at 19:03

\begin{align} p(y_t|y_{1:t-1}) &= \int p(y_t, x_t | y_{1:t-1})dx, \\ &= \int p(y_t | x_t, y_{1:t-1})p(x_t|y_{1:t-1})dx, \\ &= \int p(y_t|x_t)p(x_t|y_{1:t-1})dx. \end{align} Since we have a set of particles distributed according to $$p(x_t| y_{1:t-1})$$, we can replace the integral by a sum: $$p(y_t|y_{1:t-1}) = \frac{1}{N}\sum_{i} p(y_t|x_t^{[i]}).$$ Finally, in the bootstrap filter where the proposal distribution is chosen as the prior distribution, the weights are equal to $$p(y_t|x_t)$$.