Nonlinear filtering with unknown model parameters

Question

I have a Markov chain with scalar states $x_t$ evolving according to

$$x_t = x_{t-1} + \boldsymbol\vartheta^{\rm T} {\boldsymbol\varphi_{t-1}}(x_{t-1}) + w_t$$

with unknown but constant parameters $\boldsymbol\vartheta$, a time-dependent nonlinear "feature function" $\boldsymbol\varphi$ and Gaussian process noise $w_t$. Observations $y_t$ are obtained by

$$y_t = h(x_t) + v_t$$

with nonlinear $h$ and Gaussian measurement noise $v_t$. Both $\boldsymbol\varphi$ and $h$ can be very well locally linearized. I want to infer the true states $x_t$ as well as the unknown parameters $\boldsymbol\vartheta$. This has to be done online as $x_{t+1}$ needs to be predicted after observing $x_t$.

Here is what I've tried so far: Given the close-to-linearity of the model, I tried to infer the states using an extended Kalman filter, replacing the state transition and observation coefficients with the derivatives $$ {F}_t=1+\widehat{\boldsymbol\vartheta}^{\rm T}\frac{\partial \boldsymbol\varphi_{t-1}}{\partial x}\bigg\rvert_{\widehat{x}_{t-1}}\qquad\qquad H_t=\frac{\partial h}{\partial x}\bigg\rvert_{\widehat{x}_t}$$ with the current parameter estimate $\widehat{\boldsymbol\vartheta}$. Then, I tried two different approaches for updating $\widehat{\boldsymbol\vartheta}$:

1. Expectation-maximization: I derived an explicit formula for the parameters that optimize the log-likelihood for the true state estimates $(\widehat{x}_1,\ldots,\widehat{x}_n)$, $$\boldsymbol\vartheta^*=\left(\sum_{i=2}^n\boldsymbol\varphi(x_{i-1})\boldsymbol\varphi^{\rm T}(x_{i-1})\right)^{-1}\sum_{i=2}^n (\widehat{x}_i-\widehat{x}_{i-1})\boldsymbol\varphi_{i-1}(x_{i-1})$$ which I then used to update the parameters after filtering new data.
2. Bayesian linear regression: Treating $\widehat{x}_t-\widehat{x}_{t-1} =: d_t$ as a dependent variable of $\boldsymbol\varphi_{t-1}(\widehat{x}_{t-1})=:{\bf c}_{t-1}$, we obtain a linear model $d_t({\bf c}_{t-1})=\boldsymbol\vartheta^{\rm T}{\bf c}_{t-1}$, in which the parameters $\boldsymbol\vartheta$ can be inferred analytically from the estimated states with Bayesian multilinear regression using Gaussians.

Simulating the above procedures, I found that the Kalman filter converges if the true value of $\boldsymbol\vartheta$ is known, and the two parameter inference approaches converge if the true state $x_t$ is known. However, with both $\boldsymbol\vartheta$ and $x_t$ unknown, the synthesis of the two steps does not find the correct solution. Since I have reached the limits of my statistical aptitude at this point, I would be happy about any suggestions on how to approach this problem.

In particular:

1. What flaws could there be in the ideas outlined above? Does it seem like a sensible procedure? (Is the EM formula even correct?)
2. What other techniques could I try that might be more suitable for this problem? Is there any term to refer to this particular kind of filtering problem that I could use to find additional literature?

Answer 1

What I have changed now, using the Bayesian linear regression approach:

1. applying the Kalman filter multiple times to each new sample before regressing, feeding the filtered result back to it as input;
2. using a high variance estimate for the process noise $w_t$.

I don't know if the first part is an appropriate thing to do, but it works very well. It seems like it makes the results a little worse for near-optimal initial values, but it improves overall convergence for other values. I'm guessing that's because the extended Kalman filter can only give approximately correct results.