# Nonlinear filtering with unknown model parameters

I have a Markov chain with scalar states $$x_t$$ evolving according to

$$x_t = x_{t-1} + \boldsymbol\vartheta^{\rm T} {\boldsymbol\varphi_{t-1}}(x_{t-1}) + w_t$$

with unknown but constant parameters $$\boldsymbol\vartheta$$, a time-dependent nonlinear "feature function" $$\boldsymbol\varphi$$ and Gaussian process noise $$w_t$$. Observations $$y_t$$ are obtained by

$$y_t = h(x_t) + v_t$$

with nonlinear $$h$$ and Gaussian measurement noise $$v_t$$. Both $$\boldsymbol\varphi$$ and $$h$$ can be very well locally linearized. I want to infer the true states $$x_t$$ as well as the unknown parameters $$\boldsymbol\vartheta$$. This has to be done online as $$x_{t+1}$$ needs to be predicted after observing $$x_t$$.

Here is what I've tried so far: Given the close-to-linearity of the model, I tried to infer the states using an extended Kalman filter, replacing the state transition and observation coefficients with the derivatives $${F}_t=1+\widehat{\boldsymbol\vartheta}^{\rm T}\frac{\partial \boldsymbol\varphi_{t-1}}{\partial x}\bigg\rvert_{\widehat{x}_{t-1}}\qquad\qquad H_t=\frac{\partial h}{\partial x}\bigg\rvert_{\widehat{x}_t}$$ with the current parameter estimate $$\widehat{\boldsymbol\vartheta}$$. Then, I tried two different approaches for updating $$\widehat{\boldsymbol\vartheta}$$:

1. Expectation-maximization: I derived an explicit formula for the parameters that optimize the log-likelihood for the true state estimates $$(\widehat{x}_1,\ldots,\widehat{x}_n)$$, $$\boldsymbol\vartheta^*=\left(\sum_{i=2}^n\boldsymbol\varphi(x_{i-1})\boldsymbol\varphi^{\rm T}(x_{i-1})\right)^{-1}\sum_{i=2}^n (\widehat{x}_i-\widehat{x}_{i-1})\boldsymbol\varphi_{i-1}(x_{i-1})$$ which I then used to update the parameters after filtering new data.
2. Bayesian linear regression: Treating $$\widehat{x}_t-\widehat{x}_{t-1} =: d_t$$ as a dependent variable of $$\boldsymbol\varphi_{t-1}(\widehat{x}_{t-1})=:{\bf c}_{t-1}$$, we obtain a linear model $$d_t({\bf c}_{t-1})=\boldsymbol\vartheta^{\rm T}{\bf c}_{t-1}$$, in which the parameters $$\boldsymbol\vartheta$$ can be inferred analytically from the estimated states with Bayesian multilinear regression using Gaussians.

Simulating the above procedures, I found that the Kalman filter converges if the true value of $$\boldsymbol\vartheta$$ is known, and the two parameter inference approaches converge if the true state $$x_t$$ is known. However, with both $$\boldsymbol\vartheta$$ and $$x_t$$ unknown, the synthesis of the two steps does not find the correct solution. Since I have reached the limits of my statistical aptitude at this point, I would be happy about any suggestions on how to approach this problem.

In particular:

1. What flaws could there be in the ideas outlined above? Does it seem like a sensible procedure? (Is the EM formula even correct?)
2. What other techniques could I try that might be more suitable for this problem? Is there any term to refer to this particular kind of filtering problem that I could use to find additional literature?

2. using a high variance estimate for the process noise $$w_t$$.