3
$\begingroup$

I'm working through the derivation of the Kalman filter equations from this paper (or alternative source here) and I'm unsure of the derivation of the state prediction covariance (equation 2 in the paper).

I understand that we want $\mathbf{P}_{t|t-1}$ to be the covariance of the error of the prediction from the true value, i.e. $\text{Cov}\left(\mathbf{x}_t-\hat{\mathbf{x}}_{t|t-1}\right)$, but the author proceeds as follows:

$\mathbf{P}_{t|t-1} = \text{E}\left[\text{Cov}\left(\mathbf{x}_t-\hat{\mathbf{x}}_{t|t-1}\right)\right] = \text{E}\left[\left(\mathbf{x}_t-\hat{\mathbf{x}}_{t|t-1}\right)\left(\mathbf{x}_t-\hat{\mathbf{x}}_{t|t-1}\right)^\top\right]$

My question is, why do we need the expectation? i.e., why is this not just

$\mathbf{P}_{t|t-1} = \text{Cov}\left(\mathbf{x}_t-\hat{\mathbf{x}}_{t|t-1}\right) = \left(\mathbf{x}_t-\hat{\mathbf{x}}_{t|t-1}\right)\left(\mathbf{x}_t-\hat{\mathbf{x}}_{t|t-1}\right)^\top$

This would seem to work through the derivation in the same way to give the same end result.

Put another way, what's the expectation of a covariance?

$\endgroup$
5
  • $\begingroup$ Where does the author do this: ${P}_{t|t-1} $ $= \text{E}\left[\text{Cov}\left({x}_t-\hat{{x}}_{t|t-1}\right)\right]$ $ = \text{E}\left[\left(x_t-\hat{x}_{t|t-1}\right)\left(x_t-\hat{x}_{t|t-1}\right)^T \right]$? Sorry to be dopey, but I can't seem to spot where that is. $\endgroup$
    – Glen_b
    Jan 10, 2015 at 6:42
  • $\begingroup$ However, in the meantime note that $\left(x_t-\hat{x}_{t|t-1}\right)\left(x_t-\hat{x}_{t|t-1}\right)^T$ is a random variable. The covariance is in fact its expectation, not the random quantity you have there. see here $\endgroup$
    – Glen_b
    Jan 10, 2015 at 6:46
  • $\begingroup$ To be clear, it's the middle term in the quoted equality that I have a problem with. I can't see where it's put like that. $P_{t|t-1}=\text{E}\left[\left(x_t-\hat{x}_{t|t-1}\right)\left(x_t-\hat{x}_{t|t-1}\right)^T \right]$ is fine. $\endgroup$
    – Glen_b
    Jan 10, 2015 at 6:53
  • $\begingroup$ @Glen_b, I see, thank you. I think that I am just confused about the notation. So when we derive PCA (from here, starting on slide 13) we say that $\mathbf{W}\mathbf{X}=\mathbf{Y}$ such that the covariance matrix is diagonalised, i.e. $N\mathbf{I}=\left(\mathbf{W}\mathbf{X}\right){\left(\mathbf{W}\mathbf{X}\right)}^{\top}$ and then on slide 15 we replace $\mathbf{X}\mathbf{X}^\top$ with $\text{Cov}\left(\mathbf{X}\right)$. Why do we omit the expectation here? Is it just for clarity? $\endgroup$
    – zelanix
    Jan 10, 2015 at 15:32
  • $\begingroup$ That's a whole new question. See slide 12 for how they define covariance -- but note that's sample covariance (at least it is when $z$ is centered), so we replace expectation by an average. See that factor of 1/N? That's the averaging. $\endgroup$
    – Glen_b
    Jan 10, 2015 at 15:56

1 Answer 1

3
$\begingroup$

It looks like this is just a small confusion about the definition of covariance.

In the case of random variables, the variance-covariance matrix is an expectation:

$$\text{Cov}(X)=\operatorname{E}((X - \mu)(X - \mu)^{\operatorname{T}})$$

See here.

$\endgroup$
1
  • $\begingroup$ Thank you for clearing that up. I think that the difference between sample covariance and the covariance of a random variable is becoming clearer. $\endgroup$
    – zelanix
    Jan 11, 2015 at 3:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.