1
$\begingroup$

I struggle to implement a particle filter to evaluate the likelihood of a textbook example. I got the following process:

$x_t = \alpha + \beta x_{t-1}/(1+x_{t-1}^2) + w_t$ where $w_t \sim \mathcal{N}(0, \sigma^2)$ and

$y_t = \delta x_t + v_t$ where $v_t \sim \mathop{t}(2)$.

I want to know the likelihood $p(y|\alpha, \beta, \sigma, \delta)$ given some observations $y$ and a known $x_0 = 1$. To achieve that, I draw $N$ particles $x_t^j$ for every $x_{t}$ and calculate the density of the residuals $\hat{v}_t^j = y_t - x_t^j$. After that, I resample the particles with stratified resampling. Now $\sum_{t=1}^n\ln \left(\sum_{j=1}^N \mathop{p}(\hat{v}_t^j)/N\right)$ gives me my approximated log-likelihood.

The problem is: It works very poorly. If I deviate a little from the true parameters, the likelihood gets, as expected, smaller. However, there are always alternative constellations which give me a higher likelihood.

Let's suppose e.g. I have $n=1000$, $N = 10000$, $\alpha = 0.5$, $\beta = 0.3$, $\sigma = 1.0$ and $\delta = 1.0$. Then my likelihood is significantly lower for $\delta = 1.1$, but the overall estimate can be extremely poor (e.g. all paramters deviate by $0.5$ from their true state).

Does someone have an idea, what goes wrong in my example?

$\endgroup$

1 Answer 1

2
$\begingroup$

The problem is: It works very poorly. If I deviate a little from the true parameters, the likelihood gets, as expected, smaller. However, there are always alternative constellations which give me a higher likelihood.

This is to be expected. The approximation is just that--an approximation. The approximation's variance tends to increase for parameter constellations far away from "truth," too.

...but the overall estimate can be extremely poor

You didn't tell us how you're estimating the model. What algorithm? How are you tuning it? What's your data? This is a nontrivial estimation task.

$\endgroup$
1
  • $\begingroup$ Thanks! I used stratified resampling (clarified in link). $\endgroup$
    – JoN
    Aug 19, 2022 at 14:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.