I have a Markov chain with scalar states $x_t$ evolving according to

$$x_t = x_{t-1} + \boldsymbol\vartheta^{\rm T} {\boldsymbol\varphi_{t-1}}(x_{t-1}) + w_t$$

with unknown but constant parameters $\boldsymbol\vartheta$, a time-dependent nonlinear "feature function" $\boldsymbol\varphi$ and Gaussian process noise $w_t$. Observations $y_t$ are obtained by

$$y_t = h(x_t) + v_t$$

with nonlinear $h$ and Gaussian measurement noise $v_t$. Both $\boldsymbol\varphi$ and $h$ can be very well locally linearized. I want to infer the true states $x_t$ as well as the unknown parameters $\boldsymbol\vartheta$. This has to be done online as $x_{t+1}$ needs to be predicted after observing $x_t$.

Here is what I've tried so far: Given the close-to-linearity of the model, I tried to infer the states using an extended Kalman filter, replacing the state transition and observation coefficients with the derivatives $$ {F}_t=1+\widehat{\boldsymbol\vartheta}^{\rm T}\frac{\partial \boldsymbol\varphi_{t-1}}{\partial x}\bigg\rvert_{\widehat{x}_{t-1}}\qquad\qquad H_t=\frac{\partial h}{\partial x}\bigg\rvert_{\widehat{x}_t}$$ with the current parameter estimate $\widehat{\boldsymbol\vartheta}$. Then, I tried two different approaches for updating $\widehat{\boldsymbol\vartheta}$:

  1. Expectation-maximization: I derived an explicit formula for the parameters that optimize the log-likelihood for the true state estimates $(\widehat{x}_1,\ldots,\widehat{x}_n)$, $$\boldsymbol\vartheta^*=\left(\sum_{i=2}^n\boldsymbol\varphi(x_{i-1})\boldsymbol\varphi^{\rm T}(x_{i-1})\right)^{-1}\sum_{i=2}^n (\widehat{x}_i-\widehat{x}_{i-1})\boldsymbol\varphi_{i-1}(x_{i-1})$$ which I then used to update the parameters after filtering new data.
  2. Bayesian linear regression: Treating $\widehat{x}_t-\widehat{x}_{t-1} =: d_t$ as a dependent variable of $\boldsymbol\varphi_{t-1}(\widehat{x}_{t-1})=:{\bf c}_{t-1}$, we obtain a linear model $d_t({\bf c}_{t-1})=\boldsymbol\vartheta^{\rm T}{\bf c}_{t-1}$, in which the parameters $\boldsymbol\vartheta$ can be inferred analytically from the estimated states with Bayesian multilinear regression using Gaussians.

Simulating the above procedures, I found that the Kalman filter converges if the true value of $\boldsymbol\vartheta$ is known, and the two parameter inference approaches converge if the true state $x_t$ is known. However, with both $\boldsymbol\vartheta$ and $x_t$ unknown, the synthesis of the two steps does not find the correct solution. Since I have reached the limits of my statistical aptitude at this point, I would be happy about any suggestions on how to approach this problem.

In particular:

  1. What flaws could there be in the ideas outlined above? Does it seem like a sensible procedure? (Is the EM formula even correct?)
  2. What other techniques could I try that might be more suitable for this problem? Is there any term to refer to this particular kind of filtering problem that I could use to find additional literature?

1 Answer 1


What I have changed now, using the Bayesian linear regression approach:

  1. applying the Kalman filter multiple times to each new sample before regressing, feeding the filtered result back to it as input;
  2. using a high variance estimate for the process noise $w_t$.

I don't know if the first part is an appropriate thing to do, but it works very well. It seems like it makes the results a little worse for near-optimal initial values, but it improves overall convergence for other values. I'm guessing that's because the extended Kalman filter can only give approximately correct results.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.