4
$\begingroup$

The correction rule for Bayes filters is: $$p\left(x_{k}|D_{k}\right)=\dfrac{p\left(y_{k}|x_{k}\right)\cdot p\left(x_{k}|D_{k-1}\right)}{p\left(y_{k}|D_{k-1}\right)} $$

For:

  • State at time $k$ is $x_{k}$
  • Observation at time $k$ is $y_{k}$,
  • Past Observations at time $k$ and earlier are $D_{k}=(y_{k},y_{k-1},...)$

I am trying to derive it. In all derivations/explinations of Bayes filters i have found, it just says for this step "Simply apply Bayes Rule.".

To my knowledge, bayes rule is: $p\left(x_{k}|y_{k}\right)=p\left(y_{k}\right)p\left(x_{k}\cap y_{k}\right)=p\left(x_{k}\right)p\left(x_{k}\cap y_{k}\right) $

So I apply it on LHS:

$$p\left(x_{k}|D_{k}\right)=\dfrac{p\left(x_{k}\cap D_{k}\right)}{p\left(D_{k}\right)}$$

Then I take a look at the right hand side:

$$RHS=\dfrac{p\left(y_{k}|x_{k}\right)\cdot p\left(x_{k}|D_{k-1}\right)}{p\left(y_{k}|D_{k-1}\right)}$$

$RHS=\dfrac{\dfrac{p\left(y_{k}\cap x_{k}\right)}{p\left(x_{k}\right)}\cdot\dfrac{p\left(x_{k}\cap D_{k-1}\right)}{p\left(D_{k-1}\right)}}{\dfrac{p\left(y_{k}\cap D_{k-1}\right)}{p\left(D_{k-1}\right)}}$ Expand Conditionals

$RHS=\dfrac{p\left(y_{k}\cap x_{k}\right)\cdot p\left(x_{k}\cap D_{k-1}\right)}{p\left(x_{k}\right)\cdot p\left(y_{k}\cap D_{k-1}\right)}$ Cancel terms

$RHS=\dfrac{p\left(y_{k}\cap x_{k}\right)\cdot p\left(x_{k}\cap D_{k-1}\right)}{p\left(x_{k}\right)\cdot p\left(D_{k}\right)}$ substitute definition of $p\left(D_{k}\right)=p\left(y_{k}\cap D_{k-1}\right)$

But then I get stuck. I don't see any where to go from here.

$\endgroup$
2
$\begingroup$

The formula$$p\left(x_{k}|D_{k}\right)=\dfrac{p\left(y_{k}|x_{k}\right)\cdot p\left(x_{k}|D_{k-1}\right)}{p\left(y_{k}|D_{k-1}\right)}$$ is Bayes' rule (or Bayes' Theorem) conditional on $D_{k-1}$. Not what you wrote as your "knowledge of bayes rule", i.e., $p\left(x_{k}|y_{k}\right)=p\left(y_{k}\right)p\left(x_{k}\cap y_{k}\right)=p\left(x_{k}\right)p\left(x_{k}\cap y_{k}\right)$ which does not hold, the lhs is the joint distribution of $x_k$ and $y_k$. The proper way to define the conditional pdf $p\left(x_{k}|y_{k}\right)$ is $$p\left(x_{k}|y_{k}\right) = \dfrac{p\left(x_{k},y_{k}\right)}{p\left(y_{k}\right)}$$ And the generic Bayes rule is the inversion of the above $$p\left(x_{k}|y_{k}\right) = \dfrac{p\left(y_k|x_{k}\right)p\left(x_{k}\right)}{p\left(y_{k}\right)}$$ (Avoid using the symbol '$\cap$' in joint densities as it only applies for discrete variables.)

The key is in the above filter formula is that it relies on the assumption that $y_k$ is independent from $D_{k-1}$ given $x_k$. \begin{align*} p(x_k|D_k) &= \frac{p(x_k,D_k)}{p(D_k)}\\ &=\frac{p(x_k,y_k,D_{k-1})}{p(D_k)}\\ &=\frac{p(y_k|x_k,D_{k-1})p(x_k,D_{k-1})}{p(D_k)}\qquad\text{conditional+marginal}\\ &=\frac{p(y_k|x_k)p(x_k,D_{k-1})}{p(D_k)}\ \qquad\text{independence assumption}\\ &=\frac{p(y_k|x_k)p(x_k|D_{k-1})p(D_{k-1})}{p(y_k,D_{k-1})}\\ &=\frac{p(y_k|x_k)p(x_k|D_{k-1})p(D_{k-1})}{p(y_k|D_{k-1})p(D_{k-1})}\\ &=\dfrac{p\left(y_{k}|x_{k}\right)p\left(x_{k}|D_{k-1}\right)}{p\left(y_{k}|D_{k-1}\right)} \end{align*} which gives the result. Once again, the independence assumption is crucial in this representation. Otherwise, $p(y_k|x_k)$ would have to be replaced with $p(y_k|x_k,D_{k-1})$.

$\endgroup$
  • 1
    $\begingroup$ I was thinking of $y_x$, and $x_k$ as being discrete. But now I realise they are not. Not at all. $\endgroup$ – Lyndon White Jan 2 '15 at 12:56
  • 1
    $\begingroup$ Ok, now that I have found time to check thought it, it clears things up. I think it would be abit clearer if you finished the proof with the final line being the result being proved. It would also have helped me it you included a definition that $p(a|b)=p(a,b)/p(b)$ But it is a pretty solid answer alround, thanks. $\endgroup$ – Lyndon White Jan 5 '15 at 0:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.