1
$\begingroup$

I am trying to follow lecture notes regarding the Kalman Filter from a course taught at Stanford. The lecture notes can be found here.

The linear Gaussian state space model (LGSSM) is introduced as \begin{align*} z_0 &\sim N(0, \Sigma_0) \\ z_t &= A z_{t-1} + w_{t-1} &&\text{ for independent } &&w_{t-1} \sim N(0,Q) &\forall t \geq 1 \\ x_t &= C z_t + v_t &&\text{ for independent } &&v_t \sim N(0, R) &\forall t \geq 0 \end{align*}

Here $z_i$ represents the hidden state at time $t=i$ and $x_j$ represents the data at time $t=j$. Also, $x_{0:k} = (x_0, x_1, \ldots, x_k)$.

In section 11.2.1 Time Update, the author stats that "last time, we leveraged the fact that we know $z_{t+1} | x_{0:t}$ will take a normal distribution..."

I imagine this detail was discussed during the actual lecture, but I was not in attendance as it dates back to 2014.

I tried to work it out for $t=1$. According to the text I should be able to show that $p(z_2 | x_0, x_1)$ is Gaussian (or at least proportional).

\begin{align} p(z_2 | x_0, x_1) &= \frac{p(z_2, x_0, x_1)}{p(x_0, x_1)} && (1) \\[1ex] &= \frac{p(x_1|z_2, x_0) \cdot p(z_2 | x_0) \cdot p(x_0)}{p(x_1|x_0) \cdot p(x_0)} && (2) \\[1ex] &= \frac{p(x_1|z_2, x_0) \cdot p(z_2 | x_0)}{p(x_1|x_0)} && (3) \\[1ex] \end{align}

I can't make anything useful out this work however. How do we know that the prediction distribution, $p(z_{t+1}| x_{0:t})$, is Gaussian?

$\endgroup$
1
$\begingroup$

I believe these two facts are what you need:

  1. $p(z_{t+1} | z_t) = p(z_{t+1} | z_t, x_{0:t}) = \mathcal N \left( z_{t+1} \;|\; Az_t, Q \right)$: this is due to the Markovian nature of LGSSMs. In particular, the future is independent of the past given the present.

  2. If $p({\bf x}) = \mathcal N (\bf x \;|\; m,\; P)$ and $p({\bf y \;|\; x}) = \mathcal N (\bf y \;|\; Hx,\: R)$, then $p({\bf y}) = \mathcal N \left( \bf Hm,\; \bf HPH^{\top} + R \right)$.

Given that the filtering distribution $p(z_t | x_{0:t})$ and the state transition distribution $p(z_{t+1} | z_t)$ are both Gaussian, and in fact,

$$p(z_t | x_{0:t}) = \mathcal N \left( z_t \;|\; \hat z_{t|t} , P_{t|t} \right)$$

$$p(z_{t+1} | z_t) = \mathcal N \left( z_{t+1} \;|\; Az_t, Q \right)$$

we can make use of the facts above to see that the prediction distribution is Gaussian.

Let

  • $p(z_t | x_{0:t})$ act as $p({\bf x})$
  • $p(z_{t+1} | z_t) = p(z_{t+1} | z_t, x_{0:t})$ act as $p({\bf y \;|\; x})$
  • $p(z_{t+1} | x_{0:t})$ act as $p({\bf y})$
  • $A$ act as $\bf H$
  • $Q$ act as $\bf R$

Then we can see that $p(z_{t+1} | x_{0:t})$ is Gaussian with mean $\hat z_{t+1|t+1} = A \hat z_{t|t}$ and covariance $P_{t+1|t+1} = A P_{t|t} A^{\top} + Q$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.