I am trying to understand the Particle Filter and the motivation to use it over the regular Sequential Importance Sampling. As far as I understand until now:

1- We try to estimate the expectation of a function $f_n:\mathbb{R}^{n} \mapsto \mathbb{R}$ under the multivariate density $\pi_{n}(x_{1:n})$, namely $E_{\pi_{n}(x_{1:n})}[f_n(x_{1:n})]$ by using $N$ samples. Typically this is a dynamic system and $n$ in $\pi_{n}(x_{1:n})$ increases in each step, for example as in the filtering density of a Hidden Markov Model.

2- In each step, we sample from an incremental proposal density like $q_{n}(x_{n}|x_{1:n-1})$. In total, we have $N$ samples from an $n$ dimensional density $q_{n}(x_{1:n}) = q_{1}(x_{1})\prod_{k=2}^{n}q_{k}(x_{k}|x_{1:k-1})$ at time $n$.

3-We calculate the estimate $\frac{1}{N}\sum_{i=1}^{N}f_n(x_{1:n}^{i})w_n(x_{1:n}^{i})$ where $w_n(x_{1:n}^{i})$ is the weight function with $w_n(x_{1:n}^{i}) = \frac{\pi_{n}(x^i_{1:n})}{q_{n}(x^i_{1:n})}$.

Now, I understand that we cannot calculate the exact optimal proposal density at each step which minimizes the estimate variance, since we have only the freedom of selecting $q_{n}(x_{n}|x_{1:n-1})$, the rest of the proposal is already fixed.

What I don't understand is, why the estimate variance must increase as the dimension $n$ increases; this is explained as the primary motivation for applying Particle Filter solutions.

In order to see that, I calculated the variance of the Monte Carlo estimate $Var[\frac{1}{N}\sum_{i=1}^{N}f_n(x_{1:n}^{i})w_n(x_{1:n}^{i})]$ which turns out to be $E_{q_{n}(x_{1:n})}[f_n^2(x_{1:n})w_n^2(x_{1:n})] - \mu_n$. ($\mu_n$ is the actual expectation we want to estimate). This means in order the variance to increase, $E_{q_{n}(x_{1:n})}[f_n^2(x_{1:n})w_n^2(x_{1:n})] - \mu_n$ must increase as a function of $n$. But as a mathematical expression I am not able to show how this is guaranteed to increase with $n$: This is a very general expression which is dependent on the form of the function $f_n$, the proposal distribution $q_n$ and original distribution $\pi_n$. So I cannot see how the variance increases with the dimension $n$ here. Am I doing something wrong in my approach and calculations or am I missing something important here? Please help me on this issue.

Thanks in advance.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.