I have the following situation:

  • A set of Kalman filters with the same model, each with its own current estimated state and state covariance.

  • A measurement with a covariance matrix expressing its uncertainty.

And the following problem: I want to decide to which KF to assign the measurement based on the likelihood of the measurement being generated by the random distribution, represented by the current state and state covariance of the KF.

I know about the Mahalanobis distance, but that doesn't take into account uncertainty covariance of the measurement, only covariance of the random distribution. I also know about the Bhattacharyya distance, but if I understand it correctly, that measures the similarity of two random distributions, and I'm not sure if that's the same thing as what I want to do.

Is there a way to calculate the likelihood of a measurement with a known uncertainty covariance matrix being generated by a random distribution? If possible, can you suggest a better metric to decide to which KF to assign the measurement?

Thank you.


2 Answers 2


You might want to consider the Kullback-Leibler (KL) "distance" metric. It is often used to compare distributions. So if the signal has come from one of KFs, one would expect the it's KL distance to be the least among all.

  • $\begingroup$ Your suggestion sounds very close to what I'm looking for (from the wikipedia description). But I'm not sure what is the difference from the Bhattacharyya metric - they seem very similar. Could you please explain the difference? $\endgroup$ Commented Oct 18, 2018 at 19:15
  • $\begingroup$ Here's an interesting link - stats.stackexchange.com/questions/130432/… From the inequality between KL and Bhattacharya distance, it seems KL might be easier to work with. Also, it has some useful properties. For example, convergence in KL implies convergence in distribution etc. $\endgroup$
    – saipk
    Commented Oct 19, 2018 at 3:46
  • $\begingroup$ I guess noone else wants to contribute and your answer is technically correct (I was just hoping to get a few more different suggestions :)), so I'll accept it. $\endgroup$ Commented Oct 19, 2018 at 14:33
  • $\begingroup$ I am sorry, but I have found out that your answer is incorrect (see my new answer). I would still like to upvote it because you set me in the correct direction, but unfortunately I can't (too low SE score). $\endgroup$ Commented Dec 12, 2018 at 11:56

After some more research regarding this topic, I have found the correct answer. It turns out that what I was trying to describe and implement is a variant of the Multiple Hypothesis Tracker (MHT) [1, 2].

Answer to my question:

The correct way to calculate likelihood of a measurement $\mathbf{z}_k$ at time step $k$ given a set of previous measurements $\textbf{Z}_{k-1} = \{ \textbf{z}_{0}, \ldots , \textbf{z}_{k-1} \}$ is (taken from wikipedia [1])

$$ p(\textbf{z}) = \prod^T_{k=0}\left( p(\textbf{z}_k), | p(\textbf{z}_{0}), \ldots , p(\textbf{z}_{k-1}) \right) = \prod^T_{k=0} \mathcal{N}\left( \textbf{z}_k; \textbf{H}_k \hat{\textbf{x}}_{k|k-1}, \textbf{S}_k \right), $$

where $\hat{\textbf{x}}_{k|k-1}$ is the predicted state vector of the Kalman filter and $\textbf{H}_k$ is the matrix, mapping states of the KF to measurements (part of the system model). $\mathbf{S}_k$ is the innovation covariance, which is calculated as part of the Kalman filter update step.

This can be calculated iteratively like the Kalman filter, but it is not very numerically stable, which is why a log-likelihood $l_k = \log{p(\mathbf{z}_k)}$ is usually used. The iterative update equation for the log-likelihood is (again from wikipedia [1])

$$ l_k = l_{k-1} - \frac{1}{2}\left( \tilde{\mathbf{y}}_k^T \mathbf{S}_k^{-1} \tilde{\mathbf{y}}_k + \log{|\mathbf{S}_k|} + d_y\log{2\pi} \right), $$

where $\tilde{\mathbf{y}}_k$ is the innovation vector, which is calculated as part of the KF update step, and $d_y$ is number of dimensions of the measurement.

Notes regarding using other metrics:

The metrics for measuring a distance of distributions (such as the Mahalanobis distance or Kullback-Leibler divergence) are not well suitable for this problem, since they usually describe the 'similarity' of two random distributions, whereas in this case it is desired to express the likelihood of a measurement being generated by a stochastic system. They do not take into account the measurement model, but only the two random distributions.

Further reading

For more information on this topic, I can recommend the Wikipedia article on Kalman Filter [1] and a great summary of multitarget tracking algorithms and data association problems [2].

[1]: https://en.wikipedia.org/wiki/Kalman_filter#Marginal_likelihood

[2]: The Probabilistic Data Association Filter ESTIMATION IN THE PRESENCE OF MEASUREMENT ORIGIN UNCERTAINTY, Yaakov Bar-shalom and Fred Daum and Jim Huang, 2009


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.