3
$\begingroup$

I am trying to understand the Particle Filter and the motivation to use it over the regular Sequential Importance Sampling. As far as I understand until now:

1- We try to estimate the expectation of a function $f_n:\mathbb{R}^{n} \mapsto \mathbb{R}$ under the multivariate density $\pi_{n}(x_{1:n})$, namely $E_{\pi_{n}(x_{1:n})}[f_n(x_{1:n})]$ by using $N$ samples. Typically this is a dynamic system and $n$ in $\pi_{n}(x_{1:n})$ increases in each step, for example as in the filtering density of a Hidden Markov Model.

2- In each step, we sample from an incremental proposal density like $q_{n}(x_{n}|x_{1:n-1})$. In total, we have $N$ samples from an $n$ dimensional density $q_{n}(x_{1:n}) = q_{1}(x_{1})\prod_{k=2}^{n}q_{k}(x_{k}|x_{1:k-1})$ at time $n$.

3-We calculate the estimate $\frac{1}{N}\sum_{i=1}^{N}f_n(x_{1:n}^{i})w_n(x_{1:n}^{i})$ where $w_n(x_{1:n}^{i})$ is the weight function with $w_n(x_{1:n}^{i}) = \frac{\pi_{n}(x^i_{1:n})}{q_{n}(x^i_{1:n})}$.

Now, I understand that we cannot calculate the exact optimal proposal density at each step which minimizes the estimate variance, since we have only the freedom of selecting $q_{n}(x_{n}|x_{1:n-1})$, the rest of the proposal is already fixed.

What I don't understand is, why the estimate variance must increase as the dimension $n$ increases; this is explained as the primary motivation for applying Particle Filter solutions.

In order to see that, I calculated the variance of the Monte Carlo estimate $Var[\frac{1}{N}\sum_{i=1}^{N}f_n(x_{1:n}^{i})w_n(x_{1:n}^{i})]$ which turns out to be $E_{q_{n}(x_{1:n})}[f_n^2(x_{1:n})w_n^2(x_{1:n})] - \mu_n$. ($\mu_n$ is the actual expectation we want to estimate). This means in order the variance to increase, $E_{q_{n}(x_{1:n})}[f_n^2(x_{1:n})w_n^2(x_{1:n})] - \mu_n$ must increase as a function of $n$. But as a mathematical expression I am not able to show how this is guaranteed to increase with $n$: This is a very general expression which is dependent on the form of the function $f_n$, the proposal distribution $q_n$ and original distribution $\pi_n$. So I cannot see how the variance increases with the dimension $n$ here. Am I doing something wrong in my approach and calculations or am I missing something important here? Please help me on this issue.

Thanks in advance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.