0
$\begingroup$

In Kalman filter, we can show it's a minimum variance filter, which I believe is due to the linearity of system and the Gaussianity of noise. It comes to me that what is the optimality criterion used in Bayesian filtering, from which we can derive Kalman filter. I usually treated the optimality of Bayesian filter as granted because it's based on Bayes theorem, which in turn is the foundation of Bayesian statistics. However, I'm not sure if we can have a more concrete metric, such as mean square error or other metrics, that can be used to show that Bayesian filtering is optimizing such a metric.

I searched online and found the paper "Bayesian Filtering: From Kalman Filters to Particle Filters, and Beyond ". However, I'm confused about the following statement from the paper: enter image description here

The first part of the sentence says MMSE (minimum mean squared error) is the optimality criterion used for Bayesian filtering. But I cannot see how the posterior distribution from Bayesian filtering minimizes a mean squared error. So I feel this sentence says if we want a point estimate, we obtain MMSE (which is the conditional mean) from the posterior distribution obtained by Bayesian filtering. Is this correct understanding?

The second part of the sentence seems quite vague to me. It seems imply that, since we are using Bayes theorem to obtain the posterior distribution, we can feel assured that we are doing things in an optimal way. Does this sound correct?

$\endgroup$
  • $\begingroup$ The first part of the sentence doesn't say "MMSE (minimum mean squared error) is the optimality criterion used for Bayes filtering." It says "the Bayes risk of MMSE..", which is not the same thing. The answer to stats.stackexchange.com/questions/256218/… may be helpful in explaining what Bayes risk is. $\endgroup$ – jbowman Dec 24 '19 at 23:45
  • $\begingroup$ @jbowman Thanks for pointing this out. To clarify, can I say that the first part of the sentence says that the optimal estimator used in Bayes filtering is the Bayes risk of MMSE, and to compute this optimal estimator requires the computation of posterior distribution using Bayes filtering? $\endgroup$ – shionlau Dec 30 '19 at 15:44
0
$\begingroup$

Hi: The classical KF ( non-bayesian one ) update minimizes the mean square error of the estimate of the state WHEN both the observational noise and state noise are gaussian AND the transition equations ( F in the observation equation and the G in the state ) are linear.

Now, under the same assumptions, the bayesian filter ends up being identical to the classical so it's optimal in the MSE sense also.

But, if the noise terms aren't gaussian, then the bayesian framework will give you a posterior ( possibly, if it's tractable ) but what you want to do with that posterior as far as obtaining an estimate of the state is up to you because it depends on your optimality criteria. Maybe you want to minimize MSE or minimize absolute error or minimize something else. So, in the bayesian framework, you have a choice in terms of what you want to do with the posterior.

I think the confusion in the bayesian case stems from the fact that, because the gaussian is conjugate, then, with the distributional assumptions imposed ( both noise terms gaussian ), if you want to minimize MSE, then you obtain the same update in the bayesian case as you do in the non-bayesian case.

In the case, where you don't have linearity ( but still have gassian noises ), then you have to use the EKF and I would imagine that the optimality criteria must depend on how well the linearization approximates the non-linearity ? I'm not familiar enough with EKF to say anything useful about that case.

Still, I hope that helps. I only glanced but below looks like a decent explanation.

https://people.csail.mit.edu/mrub/talks/filtering.pdf

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.