\begin{equation} \begin{split} p(x_{1:t}|y_{1:t})& = \frac{p(y_t|x_{t}) p(x_{t}|x_{t-1}) p(x_{1:t-1}|y_{1:t-1})}{p(y_t|y_{1:t-1})} \end{split} \end{equation}

\begin{align} \label{eq:filtered_recursive} p(x_{t}|y_{1:t})& =\frac{p(y_t|x_{t}) p(x_{t}|y_{1:t-1}) }{p(y_t|y_{1:t-1})} \end{align}

I want to clear up the information I have about joint smoothing $p(x_{1:t}|y_{1:t})$ and filtering $p(x_t|y_{1:t})$ in both the equations above. So, I understand that both of these are susceptible to degeneracy. We resample to avoid degeneracy by placing particles in regions of higher likelihood. However, resampling is bad because the paths are said to degenerate. But filtering tends to contain the degeneracy of particles only to time t, where at time t+1 the filter is rejuvenated. Therefore, joint smoothing has it worse than filtering in terms of carrying the degeneracy along. Note: when I mention paths I am referring to joint smoothing and when I mention particles I am referring to filtering. Say we have a finite time series from $t=1,\dots, 10$.

When performing joint smoothing, we generate $N$ paths of $x_{1:2}, x_{1:3}, \dots x_{1:10}$ in that sequence. If we wanted the filtered estimate, all we had to do were consider the $N$ particles of the most recent time. For example, if we generated $N$ paths of $x_{1:2}$, we can obtain the filtered approximation by ignoring $x_1$ and consider $x_{2}$; i.e. we will have $N$ particles representing $x_2$. At each extension of the path, the paths are resampled. We find that when we have $N$ paths of $x_{1:10}$ that the paths coalesce for earlier times much like the figure below. As you can see, the further the filter goes the earlier times tend to coalesce.

I have two questions:

  1. Why is this degeneracy bad? It was written somewhere that it is because the paths are a bad approximations of the joint $p(x_{1:t}|y_{1:t})$ . Visually I have simulated this and I found that actually if you compute the mean of all the paths, they tend to trace the state very well, especially at lower times when the paths coalesce. It can't be a matter of variance as well because If I re-run my filter, I always get the same answer with extremely small variance at the times when the paths coalesce. Why is the degeneracy bad when it is estimating my process almost perfectly at points where the paths coalesce.
  2. Comparing joint smooth and filtered . My thinking is that I can compare the estimation of the joint smooth versus the filtered by doing this. Suppose I ran my filter to get paths $x_{1:2}$, the particles of $x_2$ represents the estimation for my filtered at time 2. Then I continue to run my filter to get paths $x_{1:10}$. I consider the particles $x_2$ from $x_{1:10}$(after all the degeneration has happened). Isn't it that the latter estimate $x_2$ case is meant to be worse because of the degeneration. Or is this the wrong way to think about it?

There exist this jargon about how joint smoothing is more unstable than particle filters because the asymptotic variance is unbounded with increasing time , whereas particle filters are bounded. It would be nice if someone can explain in that context.

enter image description here enter image description here

  • 1
    $\begingroup$ Say you wanted to estimate $E[x_1|y_{1:25}]$. You would have effectively two data points for this. That's high variance. $\endgroup$
    – Taylor
    Commented Apr 2, 2017 at 2:01
  • $\begingroup$ @Taylor Ahh I know what I have been doing wrong. Assume I calculate my statistics after resampling, I have been computing $\frac{\sqrt(\sigma^2)}{\sqrt{N}}$ with my replicated particles as oppose to unique particles. The variance of the mean drops because of the $\sqrt{N}$ term when considering a further time. The variance of the particles however increases. So degeneracy is bad because of the high variance of the mean or some expected value for earlier times?. $\endgroup$ Commented Apr 2, 2017 at 9:55
  • $\begingroup$ Actually, what I said is wrong. There is no evidence of a decrease in variance of the mean at earlier times. When I simulate it should not be by repeated runs of the experiment right? I only take the sample variance of my particles at the time point I am interested in. $\endgroup$ Commented Apr 2, 2017 at 10:31


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.