Say the observations are $x_i$ and the states are $y_i$ in a sequential model.
I understand that particle filtering works by generating "particles" from $p(y_i | x_1,\ldots,x_i)$ for approximating $p(y_{i+1} | x_1,\ldots,x_{i+1})$.
How do we decide on how many "particles" to use as we go along on the chain? Do we choose a fixed number in the beginning, and stick to it (one that works well experimentally), or do we change the number of particles used as the particle filtering algorithm proceeds?
For this choice I often think about the trade-off between computational cost and the variance of the resulting estimates. As you increase the number of particles or sample size the former increase, while the latter decreases.
Often I do a simple computational experiment:
I create a grid of potential numbers of particle (say $10^2$, $10^3$ and $10^4$).
I do the filtering $N$ times using each sample size.
I plot the sample variance of the quantity I'm interested in (for example the variance of the estimated likelihood) on the Y axis, with the number of particles on the X axis.
You should get a convex curve, that becomes flat as the number of particles increases. Generally I just look at it, and choose a number of particle that seems reasonable in the sense that increasing the number of particles further wouldn't reduce the variance by much.
Obviously this is just a practical approach, maybe there are more rigorous ways of looking at the problem.
